DoctorGPT (A Decoder-Only Implementation of GPT2 Architecture)

November 2023
Vol-9, Issue-6
Paper ID: 22101
ISSN: 2395-4396
Downloads: 64
Download PDF

Abstract & Details

Research Area
Artificial Intelligence
Keywords
Natural Language Processing (NLP) Decoder-Only Transformer Text Generation Deep Learning PyTorch Autoregressive Generation Self-Attention Mechanism Benchmark Datasets Model Evaluation Long-Range Dependencies BLEU Score Perplexity Machine Translation Text Summarization Creative Writing
Abstract
Natural language processing (NLP) has developed dramatically in recent years, particularly with the introduction of transformer-based architectures in deep learning. Among these, decoder-only transformers stand out as a straightforward yet promising approach to text generation. This project focuses on developing and testing a decoder-only transformer model for text creation jobs. A crucial part of NLP is text creation, which includes synthesizing human-like text from provided inputs. Traditional techniques, like as n-gram models, have limitations in terms of understanding long-distance relationships and producing cohesive text. However, decoder-only transformers that use self-attention and autoregressive generation have demonstrated superior text creation capabilities. The key goals here are to build a decoder-only transformer model with PyTorch, train it on large amounts of text input, and evaluate its ability to generate high-quality text. Data cleansing, creating the model's architecture, training with optimization approaches, and evaluating performance using measures such as perplexity and BLEU score are all part of the process. The results show the model's ability to generate coherent and contextually relevant content, demonstrating its promise in machine translation, text summarization, and creative writing.

Author Information

# Name Institute / Affiliation
1 Jeethu Srinivas A Bannari Amman Institute of Technology
2 Hari Prechetha K Bannari Amman Institute of Technology
3 Kavin Devraj Bannari Amman Institute of Technology

How to Cite

Use the following formats to cite this article in your research.

APA Style
A, Jeethu Srinivas, K, Hari Prechetha, & Devraj, Kavin (2023). DoctorGPT (A Decoder-Only Implementation of GPT2 Architecture). International Journal of Advance Research and Innovative Ideas In Education, 9(6), 1002-1008.
MLA Style
A, Jeethu Srinivas, et al. "DoctorGPT (A Decoder-Only Implementation of GPT2 Architecture)." International Journal of Advance Research and Innovative Ideas In Education, vol. 9, no. 6, 2023, pp. 1002-1008.
IEEE Style
Jeethu Srinivas A, Hari Prechetha K, and Kavin Devraj, "DoctorGPT (A Decoder-Only Implementation of GPT2 Architecture)," International Journal of Advance Research and Innovative Ideas In Education, vol. 9, no. 6, pp. 1002-1008, 2023.
Vancouver Style
A Jeethu Srinivas, K Hari Prechetha, Devraj Kavin. DoctorGPT (A Decoder-Only Implementation of GPT2 Architecture). International Journal of Advance Research and Innovative Ideas In Education. 2023;9(6):1002-1008.
Harvard Style
A, Jeethu Srinivas, K, Hari Prechetha, & Devraj, Kavin (2023) 'DoctorGPT (A Decoder-Only Implementation of GPT2 Architecture)', International Journal of Advance Research and Innovative Ideas In Education, 9(6), pp. 1002-1008.
Chicago Style
A, Jeethu Srinivas, Hari Prechetha K, and Kavin Devraj. "DoctorGPT (A Decoder-Only Implementation of GPT2 Architecture)." International Journal of Advance Research and Innovative Ideas In Education 9, no. 6 (2023): 1002-1008.
Turabian Style
A, Jeethu Srinivas, Hari Prechetha K, and Kavin Devraj. "DoctorGPT (A Decoder-Only Implementation of GPT2 Architecture)." International Journal of Advance Research and Innovative Ideas In Education 9, no. 6 (2023): 1002-1008.

Export Citation

Related Research

AI-Based Smart Waste Segregation and Recycling Assistant
Priyanka Patil et al. 2026 Artificial Intelligence (AI) and Machine Learning
Download PDF
Precision Agriculture Using Computer Vision: A Review of In-Field Disease Detection, Foliar Severity Estimation
Ashlesha Adsul et al. 2026 Precision Agriculture, Computer Vision, AI, Machine Learning
Download PDF
Explainable AI for Classification and Prioritization of QHSE Audit Findings
Dr. Feroz Khan Ismail 2026 Artificial Intelligence, Quality Management, QHSE Auditing, Risk Management and Engineering
Download PDF
Development of a Custom Handwritten Kannada Dataset and CNN-BiLSTM-CTC Based OCR System
Nandini G R et al. 2026 OCR, Machine learning, Image Processing, CNN, Deep learning
Download PDF
CodeCoach AI: A Review of Automated Programming Assessment, Explainable Feedback, and an AI-Assisted Coding Practice Framework
Mr. Pratham Tembhurne et al. 2026 Artificial Intelligence in Education, focusing on automated code assessment and personalized feedback to improve programming skills.
Download PDF