Unlock the Potential of Open AI in Smart Manufacturing Transformers were a generative AI breakthrough :
A transformer model is a type of neural network that learns context and meaning by tracking relationships in sequential data , such as words in sentences , pixels in images , or notes in music .
They are scalable , which means their performance and accuracy improve as they are made larger and fed more data . But more importantly , transformer models can be trained through selfsupervised or unsupervised learning , meaning they require no or very little human-annotated data .
Figure
2-2 : Transformer architecture 2 .
Transformers can detect trends and anomalies to prevent fraud , streamline manufacturing , make online recommendations , generate initial code drafts , code correction and refactoring , rootcause analysis , and generate new system designs .
People use transformers every time they search on Google or Microsoft Bing . OpenAI advanced the transformer approach with GPT-2 and GPT-3 ( GPT - Generative Pretrained Transformer ).
2.1 THE GENERATIVE AI FOUNDATION MODEL
A foundation model is a machine learning model that is trained on a large amount of unlabeled data , allowing it to be adapted to a wide variety of tasks . These models capture general patterns and structure of data , and aim to replace task-specific models by unifying different tasks and modalities into one , eliminating the need to train individual models and integrate several models together . An example is a large language foundation model that can perform various NLP tasks and can be used out-of-the-box due to its extensive training .
2 https :// machinelearningmastery . com / the-transformer-model / Journal of Innovation 47