Transformers for Natural Language Processing and Computer Vision: Explore Generative AI and Large Language Models with Hugging Face, ChatGPT, GPT-4V, and DALL-E 3, 3rd Edition / : Hugging Face, Chat GPT, GPT-4V DALE 3., 3-
: 2024
: Rothman Denis /
: Packt Publishing
ISBN: 978-1-80512-872-4
: Expert Insight
:
: PDF/EPUB
: (eBook)
:
: 731
: Unleash the full potential of transformers with this comprehensive guide covering architecture, capabilities, risks, and practical implementations on OpenAI, Google Vertex AI, and Hugging Face
Key Features:
Master NLP and vision transformers, from the architecture to fine-tuning and implementation
Learn how to apply Retrieval Augmented Generation (RAG) with LLMs using customized texts and embeddings
Mitigate LLM risks, such as hallucinations, using moderation models and knowledge bases
Book Description:
Transformers for Natural Language Processing and Computer Vision, Third Edition, explores Large Language Models (LLMs) architectures, applications, and various platforms (Hugging Face, OpenAI, and Google Vertex AI) used for Natural Language Processing (NLP) and Computer Vision (CV).
The book guides you through different transformer architectures to the latest Foundation Models and Generative AI. Youll pretrain and fine-tune LLMs and work through different use cases, from summarization to implementing question-answering systems with embedding-based search techniques. This book explains the risks of LLMs, from hallucinations and memorization to privacy, and how to mitigate risks using moderation models with rule and knowledge bases. Youll implement Retrieval Augmented Generation (RAG) with LLMs to improve the accuracy of your models and give you greater control over LLM outputs.
Dive into generative vision transformers and multimodal model architectures and build applications, such as image and video-to-text classifiers. Go further by combining different models and platforms and learning about AI agent replication.
This book provides you with an understanding of transformer architectures, pretraining, fine-tuning, LLM use cases, and best practices.
What You Will Learn:
Learn how to pretrain and fine-tune LLMs
Learn how to work with multiple platforms, such as Hugging Face, OpenAI, and Google Vertex AI
Learn about different tokenizers and the best practices for preprocessing language data
Implement Retrieval Augmented Generation and rules bases to mitigate hallucinations
Visualize transformer model activity for deeper insights using BertViz, LIME, and SHAP
Create and implement cross-platform chained models, such as HuggingGPT
Go in-depth into vision transformers with CLIP, DALL-E 2, DALL-E 3, and GPT-4V
Who this book is for:
This book is ideal for NLP and CV engineers, software developers, data scientists, machine learning engineers, and technical leaders looking to advance their LLMs and generative AI skills or explore the latest trends in the field.
Knowledge of Python and machine learning concepts is required to fully understand the use cases and code examples. However, with examples using LLM user interfaces, prompt engineering, and no-code model building, this book is great for anyone curious about the AI revolution.
transformers , , , OpenAI, Google Vertex AI Hugging Face.
K :
-
, (RAG) LLMS,
, LLM, ,
:
Transformers for Natural Language Processing and Computer Vision (" ") , (LLMS) (Hugging Face, OpenAI Google Vertex AI), (NLP) (CV).
transformer, . LLM , . , LLM, , , , . (RAG) LLMs, LLM.
, . , .
transformer, , , LLM .
:
, LLM
, , Hugging Face, OpenAI Google Vertex AI
transformer BertViz, LIME SHAP
, HuggingGPT.
vision CLIP, DALL-E 2, DALL-E 3 GPT-4V
:
CV, , , , LLM .
Python . , LLM, , , .
Preface xxv
Chapter 1: What Are Transformers? 1
Foundation Models ................................................................................................................... 3
From general-purpose to specific tasks 4
A brief history of how transformers were born .......................................................................... 8
From one token to an AI revolution 11
The new role of AI professionals ............................................................................................. 12
The future of AI professionals 14
What resources should we use? 14
Decision-making guidelines 15
The rise of seamless transformer APIs .................................................................................... 16
Choosing ready-to-use API-driven libraries 18
Choosing a cloud platform and transformer model 19
Summary ............................................................................................................................... 19
Questions ............................................................................................................................... 20
References ............................................................................................................................. 20
Further reading ...................................................................................................................... 21
Chapter 2: Getting Started with the Architecture of the Transformer Model 23
The rise of the Transformer: Attention Is All You Need ............................................................. 24
The encoder stack 26
Input embedding 28
Positional encoding 31
Sublayer 1: Multi-head attention 36
Sublayer 2: Feedforward network 51
The decoder stack 52
Output embedding and position encoding 53
The attention layers 53
The FFN sublayer, the post-LN, and the linear layer 54
Training and performance ...................................................................................................... 55
Hugging Face transformer models .......................................................................................... 55
Summary ............................................................................................................................... 56
Questions ............................................................................................................................... 57
References ............................................................................................................................. 57
Further reading ...................................................................................................................... 57
Chapter 3: Emergent vs Downstream Tasks: The Unseen Depths of Transformers 59
The paradigm shift: What is an NLP task? ................................................................................ 60
Inside the head of the attention sublayer of a transformer 61
Exploring emergence with ChatGPT 65
Investigating the potential of downstream tasks ...................................................................... 68
Evaluating models with metrics 68
Accuracy score 68
F1-score 68
MCC 69
Human evaluation 69
Benchmark tasks and datasets 70
Defining the SuperGLUE benchmark tasks 74
Running downstream tasks ..................................................................................................... 78
The Corpus of Linguistic Acceptability (CoLA) 79
Stanford Sentiment TreeBank (SST-2) 79
Microsoft Research Paraphrase Corpus (MRPC) 80
Winograd schemas 81
Summary ............................................................................................................................... 82
Questions ............................................................................................................................... 82
References ............................................................................................................................. 83
Further reading ...................................................................................................................... 83
Chapter 4: Advancements in Translations with Google Trax,
Google Translate, and Gemini 85
Defining machine translation .................................................................................................. 86
Human transductions and translations 87
Machine transductions and translations 88
Evaluating machine translations ............................................................................................. 88
Preprocessing a WMT dataset 88
Preprocessing the raw data 89
Finalizing the preprocessing of the datasets 92
Evaluating machine translations with BLEU 95
Geometric evaluations 95
Applying a smoothing technique 97
Translations with Google Trax ................................................................................................. 99
Installing Trax 99
Creating the Original Transformer model 99
Initializing the model using pretrained weights 101
Tokenizing a sentence 101
Decoding from the Transformer 102
De-tokenizing and displaying the translation 102
Translation with Google Translate ......................................................................................... 103
Translation with a Google Translate AJAX API Wrapper 104
Implementing googletrans 104
Translation with Gemini ....................................................................................................... 106
Geminis potential 106
Summary ............................................................................................................................. 107
Questions ............................................................................................................................. 108
References ........................................................................................................................... 108
Further reading .................................................................................................................... 108
Chapter 5: Diving into Fine-Tuning through BERT 111
The architecture of BERT ...................................................................................................... 112
The encoder stack 113
Preparing the pretraining input environment 115
Pretraining and fine-tuning a BERT model 118
Fine-tuning BERT ................................................................................................................. 119
Defining a goal 120
Hardware constraints 120
Installing Hugging Face Transformers 120
Importing the modules 121
Specifying CUDA as the device for torch 121
Loading the CoLA dataset 122
Creating sentences, label lists, and adding BERT tokens 124
Activating the BERT tokenizer 124
Processing the data 124
Creating attention masks 125
Splitting the data into training and validation sets 125
Converting all the data into torch tensors 125
Selecting a batch size and creating an iterator 126
BERT model configuration 127
Loading the Hugging Face BERT uncased base model 128
Optimizer grouped parameters 129
The hyperparameters for the training loop 131
The training loop 132
Training evaluation 133
Predicting and evaluating using the holdout dataset 134
Exploring the prediction process 135
Evaluating using the Matthews correlation coefficient 137
Matthews correlation coefficient evaluation for the whole dataset 137
Building a Python interface to interact with the model ........................................................... 138
Saving the model 138
Creating an interface for the trained model 139
Interacting with the model 140
Summary ............................................................................................................................. 141
Questions ............................................................................................................................. 142
References ........................................................................................................................... 143
Further reading .................................................................................................................... 143
Chapter 6: Pretraining a Transformer from Scratch through RoBERTa 145
Training a tokenizer and pretraining a transformer ............................................................... 146
Building KantaiBERT from scratch ........................................................................................ 148
Step 1: Loading the dataset 148
Step 2: Installing Hugging Face transformers 149
Step 3: Training a tokenizer 150
Step 4: Saving the files to disk 152
Step 5: Loading the trained tokenizer files 153
Step 6: Checking resource constraints: GPU and CUDA 154
Step 7: Defining the configuration of the model 155
Step 8: Reloading the tokenizer in transformers 155
Step 9: Initializing a model from scratch 155
Exploring the parameters 157
Step 10: Building the dataset 161
Step 11: Defining a data collator 162
Step 12: Initializing the trainer 162
Step 13: Pretraining the model 164
Step 14: Saving the final model (+tokenizer + config) to disk 164
Step 15: Language modeling with FillMaskPipeline 165
Pretraining a Generative AI customer support model on X data .............................................. 166
Step 1: Downloading the dataset 167
Step 2: Installing Hugging Face transformers 167
Step 3: Loading and filtering the data 167
Step 4: Checking Resource Constraints: GPU and CUDA 169
Step 5: Defining the configuration of the model 169
Step 6: Creating and processing the dataset 170
Step 7: Initializing the trainer 171
Step 8: Pretraining the model 172
Step 9: Saving the model 173
Step 10: User interface to chat with the Generative AI agent 173
Further pretraining 175
Limitations 175
Next steps ............................................................................................................................ 175
Summary ............................................................................................................................. 175
Questions ............................................................................................................................. 176
References ........................................................................................................................... 176
Further reading .................................................................................................................... 177
Chapter 7: The Generative AI Revolution with ChatGPT 179
GPTs as GPTs ........................................................................................................................ 180
Improvement 180
Diffusion 182
New application sectors 182
Self-service assistants 182
Development assistants 182
Pervasiveness 183
The architecture of OpenAI GPT transformer models ............................................................ 183
The rise of billion-parameter transformer models 184
The increasing size of transformer models 184
Context size and maximum path length 186
From fine-tuning to zero-shot models 187
Stacking decoder layers 188
GPT models 189
OpenAI models as assistants ................................................................................................. 192
ChatGPT provides source code 192
GitHub Copilot code assistant 193
General-purpose prompt examples 196
Getting started with ChatGPT GPT-4 as an assistant 197
1. GPT-4 helps to explain how to write source code 197
2. GPT-4 creates a function to show the YouTube presentation of GPT-4 by Greg Brockman on March 14,
2023 197
3. GPT-4 creates an application for WikiArt to display images 198
4. GPT-4 creates an application to display IMDb reviews 198
5. GPT-4 creates an application to display a newsfeed 199
6. GPT-4 creates a k-means clustering (KMC) algorithm 200
Getting started with the GPT-4 API ........................................................................................ 201
Running our first NLP task with GPT-4 201
Steps 1: Installing OpenAI and Step 2: Entering the API key 201
Step 3: Running an NLP task with GPT-4 201
Key hyperparameters 202
Running multiple NLP tasks 203
Retrieval Augmented Generation (RAG) with GPT-4 ............................................................... 204
Installation 204
Document retrieval 205
Augmented retrieval generation 206
Summary ............................................................................................................................. 209
Questions ............................................................................................................................. 210
References ........................................................................................................................... 210
Further reading .................................................................................................................... 210
Chapter 8: Fine-Tuning OpenAI GPT Models 213
Risk management ................................................................................................................. 214
Fine-tuning a GPT model for completion (generative) ............................................................ 215
1. Preparing the dataset ........................................................................................................ 217
1.1. Preparing the data in JSON 217
1.2. Converting the data to JSONL 219
2. Fine-tuning an original model ........................................................................................... 221
3. Running the fine-tuned GPT model .................................................................................... 224
4. Managing fine-tuned jobs and models ................................................................................ 227
Before leaving ...................................................................................................................... 228
Summary ............................................................................................................................. 229
Questions ............................................................................................................................. 230
References ........................................................................................................................... 230
Further reading .................................................................................................................... 230
Chapter 9: Shattering the Black Box with Interpretable Tools 233
Transformer visualization with BertViz ................................................................................. 234
Running BertViz 235
Step 1: Installing BertViz and importing the modules 235
Step 2: Load the models and retrieve attention 235
Step 3: Head view 236
Step 4: Processing and displaying attention heads 237
Step 5: Model view 238
Step 6: Displaying the output probabilities of attention heads 239
Streaming the output of the attention heads 240
Visualizing word relationships using attention scores with pandas 242
exBERT 245
Interpreting Hugging Face transformers with SHAP .............................................................. 246
Introducing SHAP 246
Explaining Hugging Face outputs with SHAP 249
Transformer visualization via dictionary learning ................................................................. 251
Transformer factors 251
Introducing LIME 253
The visualization interface 254
Other interpretable AI tools .................................................................................................. 256
LIT 257
PCA 257
Running LIT 258
OpenAI LLMs explain neurons in transformers 260
Limitations and human control 264
Summary ............................................................................................................................. 264
Questions ............................................................................................................................. 265
References ........................................................................................................................... 265
Further reading .................................................................................................................... 266
Chapter 10: Investigating the Role of Tokenizers in
Shaping Transformer Models 267
Matching datasets and tokenizers ......................................................................................... 268
Best practices 268
Step 1: Preprocessing 269
Step 2: Quality control 270
Step 3: Continuous human quality control 270
Word2Vec tokenization 272
Case 0: Words in the dataset and the dictionary 275
Case 1: Words not in the dataset or the dictionary 276
Case 2: Noisy relationships 278
Case 3: Words in a text but not in the dictionary 278
Case 4: Rare words 279
Case 5: Replacing rare words 280
Exploring sentence and WordPiece tokenizers to understand the efficiency
of subword tokenizers for transformers ................................................................................ 280
Word and sentence tokenizers 280
Sentence tokenization 282
Word tokenization 282
Regular expression tokenization 282
Treebank tokenization 283
White space tokenization 283
Punkt tokenization 284
Word punctuation tokenization 284
Multi-word tokenization 284
Subword tokenizers 285
Unigram language model tokenization 286
SentencePiece 287
Byte-Pair Encoding (BPE) 288
WordPiece 289
Exploring in code 289
Detecting the type of tokenizer 289
Displaying token-ID mappings 291
Analyzing and controlling the quality of token-ID mappings 292
Summary ............................................................................................................................. 294
Questions ............................................................................................................................. 295
References ........................................................................................................................... 295
Further reading .................................................................................................................... 295
Chapter 11: Leveraging LLM Embeddings as an Alternative to Fine-Tuning 297
LLM embeddings as an alternative to fine-tuning ................................................................... 298
From prompt design to prompt engineering 299
Fundamentals of text embedding with NLKT and Gensim ...................................................... 299
Installing libraries 299
1. Reading the text file 300
2. Tokenizing the text with Punkt 300
Preprocessing the tokens 300
3. Embedding with Gensim and Word2Vec 302
4. Model description 303
5. Accessing a word and vector 305
6. Exploring Gensims vector space 306
7. TensorFlow Projector 309
Implementing question-answering systems with embedding-based search techniques ........... 313
1. Installing the libraries and selecting the models 313
2. Implementing the embedding model and the GPT model 314
2.1 Evaluating the model with a knowledge base: GPT can answer questions 315
2.2 Add a knowledge base 316
2.3 Evaluating the model without a knowledge base: GPT cannot answer questions 316
3. Prepare search data 317
4. Search 319
5. Ask 320
5.1.Example question 322
5.2.Troubleshooting wrong answers 322
Transfer learning with Ada embeddings ................................................................................ 324
1. The Amazon Fine Food Reviews dataset 324
1.2. Data preparation 325
2. Running Ada embeddings and saving them for future reuse 326
3. Clustering 327
3.1. Find the clusters using k-means clustering 328
3.2. Display clusters with t-SNE 329
4. Text samples in the clusters and naming the clusters 330
Summary ............................................................................................................................. 332
Questions ............................................................................................................................. 332
References ........................................................................................................................... 333
Further reading .................................................................................................................... 333
Chapter 12: Toward Syntax-Free Semantic Role Labeling with ChatGPT and GPT-4
335
Getting started with cutting-edge SRL ................................................................................... 336
Entering the syntax-free world of AI ...................................................................................... 338
Defining SRL ........................................................................................................................ 338
Visualizing SRL 339
SRL experiments with ChatGPT with GPT-4 ........................................................................... 339
Basic sample 340
Difficult sample 343
Questioning the scope of SRL ................................................................................................ 344
The challenges of predicate analysis 344
Redefining SRL ..................................................................................................................... 346
From task-specific SRL to emergence with ChatGPT ............................................................... 348
1. Installing OpenAI 348
2. GPT-4 dialog function 349
3. SRL 349
Sample 1 (basic) 350
Sample 2 (basic) 351
Sample 3 (basic) 352
Sample 4 (difficult) 353
Sample 5 (difficult) 354
Sample 6 (difficult) 355
Summary ............................................................................................................................. 355
Questions ............................................................................................................................. 356
References ........................................................................................................................... 356
Further reading .................................................................................................................... 357
Chapter 13: Summarization with T5 and ChatGPT 359
Designing a universal text-to-text model ............................................................................... 360
The rise of text-to-text transformer models ........................................................................... 361
A prefix instead of task-specific formats ................................................................................ 362
The T5 model ....................................................................................................................... 363
Text summarization with T5 ................................................................................................. 365
Hugging Face 365
Selecting a Hugging Face transformer model 365
Initializing the T5-large transformer model 367
Getting started with T5 367
Exploring the architecture of the T5 model 369
Summarizing documents with T5-large 372
Creating a summarization function 373
A general topic sample 374
The Bill of Rights sample 376
A corporate law sample 377
From text-to-text to new word predictions with OpenAI ChatGPT ........................................... 379
Comparing T5 and ChatGPTs summarization methods 379
Pretraining 379
Specific versus non-specific tasks 380
Summarization with ChatGPT 380
Summary ............................................................................................................................. 384
Questions ............................................................................................................................. 385
References ........................................................................................................................... 385
Further reading .................................................................................................................... 386
Chapter 14: Exploring Cutting-Edge LLMs with Vertex AI and PaLM 2 387
Architecture ......................................................................................................................... 388
Pathways 388
Client 390
Resource manager 390
Intermediate representation 390
Compiler 390
Scheduler 390
Executor 391
PaLM 391
Parallel layer processing that increases training speed 391
Shared input-output embeddings, which saves memory 392
No biases, which improves training stability 392
Rotary Positional Embedding (RoPE) improves model quality 392
SwiGLU activations improve model quality 392
PaLM 2 393
Improved performance, faster, and more efficient 393
Scaling laws, optimal model size, and the number of parameters 394
State-of-the-art (SOA) performance and a new training methodology 394
Assistants ............................................................................................................................. 394
Gemini 396
Google Workspace 396
Google Colab Copilot 399
Vertex AI PaLM 2 interface 401
Vertex AI PaLM 2 assistant 404
Vertex AI PaLM 2 API ............................................................................................................ 407
Question answering 408
Question-answer task 409
Summarization of a conversation 410
Sentiment analysis 412
Multi-choice problems 414
Code 416
Fine-tuning .......................................................................................................................... 420
Creating a bucket 421
Fine-tuning the model 422
Summary ............................................................................................................................. 424
Questions ............................................................................................................................. 424
References ........................................................................................................................... 425
Further reading .................................................................................................................... 425
Chapter 15: Guarding the Giants: Mitigating Risks in Large Language Models 427
The emergence of functional AGI .......................................................................................... 428
Cutting-edge platform installation limitations ....................................................................... 430
Auto-BIG-bench .................................................................................................................... 433
WandB ................................................................................................................................. 439
When will AI agents replicate? .............................................................................................. 441
Function: `create_vocab` 442
Process: 442
Function: `scrape_wikipedia` 442
Process: 442
Function: `create_dataset` 442
Process: 442
Classes: `TextDataset`, `Encoder`, and `Decoder` 442
Function: `count_parameters` 442
Function: `main` 442
Process: 443
Saving and Executing the Model 443
Risk management ................................................................................................................. 443
Hallucinations and memorization 444
Memorization 448
Risky emergent behaviors 448
Disinformation 450
Influence operations 451
Harmful content 453
Privacy 455
Cybersecurity 455
Risk mitigation tools with RLHF and RAG .............................................................................. 456
1. Input and output moderation with transformers and a rule base 457
2. Building a knowledge base for ChatGPT and GPT-4 461
Adding keywords 462
3. Parsing the user requests and accessing the KB 463
4. Generating ChatGPT content with a dialog function 464
Token control 466
Moderation 466
Summary ............................................................................................................................. 466
Questions ............................................................................................................................. 467
References ........................................................................................................................... 467
Further reading .................................................................................................................... 467
Chapter 16: Beyond Text: Vision Transformers in the Dawn of Revolutionary AI 469
From task-agnostic models to multimodal vision transformers .............................................. 470
ViT Vision Transformer ...................................................................................................... 471
The basic architecture of ViT 471
Step 1: Splitting the image into patches 472
Step 2: Building a vocabulary of image patches 472
Step 3: The transformer 473
Vision transformers in code 474
A feature extractor simulator 475
The transformer 478
Configuration and shapes 479
CLIP .................................................................................................................................... 484
The basic architecture of CLIP 484
CLIP in code 485
DALL-E 2 and DALL-E 3 ......................................................................................................... 489
The basic architecture of DALL-E 489
Getting started with the DALL-E 2 and DALL-E 3 API 490
Creating a new image 491
Creating a variation of an image 492
From research to mainstream AI with DALL-E 493
GPT-4V, DALL-E 3, and divergent semantic association ........................................................... 496
Defining divergent semantic association 496
Creating an image with ChatGPT Plus with DALL-E 497
Implementing the GPT-4V API and experimenting with DAT 500
Example 1: A standard image and text 500
Example 2: Divergent semantic association, moderate divergence 502
Example 3: Divergent semantic association, high divergence 503
Summary ............................................................................................................................. 505
Questions ............................................................................................................................. 506
References ........................................................................................................................... 506
Further Reading ................................................................................................................... 506
Chapter 17: Transcending the Image-Text Boundary with Stable Diffusion 509
Transcending image generation boundaries ......................................................................... 510
Part I: Defining text-to-image with Stable Diffusion ............................................................... 512
1. Text embedding using a transformer encoder 512
2. Random image creation with noise 514
3. Stable Diffusion model downsampling 514
4. Decoder upsampling 516
5. Output image 517
Running the Keras Stable Diffusion implementation 517
Part II: Running text-to-image with Stable Diffusion .............................................................. 519
Generative AI Stable Diffusion for a Divergent Association Task (DAT) 521
Part III: Video ...................................................................................................................... 522
Text-to-video with Stability AI animation 522
Text-to-video, with a variation of OpenAI CLIP 525
A video-to-text model with TimeSformer 526
Preparing the video frames 527
Putting the TimeSformer to work to make predictions on the video frames 529
Summary ............................................................................................................................. 530
Questions ............................................................................................................................. 531
References ........................................................................................................................... 531
Further reading .................................................................................................................... 531
Chapter 18: Hugging Face AutoTrain: Training Vision Models without Coding 533
Goal and scope of this chapter .............................................................................................. 534
Getting started ..................................................................................................................... 535
Uploading the dataset ........................................................................................................... 536
No coding? 539
Training models with AutoTrain ............................................................................................ 539
Deploying a model ............................................................................................................... 541
Running our models for inference ........................................................................................ 543
Retrieving validation images 543
The program will now attempt to classify the validation images. We will see how a vision transformer
reacts to this image. 545
Inference: image classification 545
Validation experimentation on the trained models 547
ViTForImageClassification 548
SwinForImageClassification 1 551
BeitForImage Classification 553
SwinForImageClassification 2 556
ConvNextForImageClassification 558
ResNetForImageClassification 560
Trying the top ViT model with a corpus 563
Summary ............................................................................................................................. 564
Questions ............................................................................................................................. 565
References ........................................................................................................................... 566
Further reading .................................................................................................................... 566
Chapter 19: On the Road to Functional AGI with HuggingGPT and its Peers 567
Defining F-AGI ..................................................................................................................... 569
Installing and importing ....................................................................................................... 571
Validation set ....................................................................................................................... 571
Level 1 image: easy 571
Level 2 image: difficult 572
Level 3 image: very difficult 573
HuggingGPT ......................................................................................................................... 574
Level 1: Easy 575
Level 2: Difficult 578
Level 3: Very difficult 580
CustomGPT .......................................................................................................................... 583
Google Cloud Vision 584
Level 1: Easy 585
Level 2: Difficult 587
Level 3: Very difficult 588
Model chaining: Chaining Google Cloud Vision to ChatGPT 590
Model Chaining with Runway Gen-2 ...................................................................................... 593
Midjourney: Imagine a ship in the galaxy 593
Gen-2: Make this ship sail the sea 594
Summary ............................................................................................................................. 595
Questions ............................................................................................................................. 596
References ........................................................................................................................... 596
Further Reading ................................................................................................................... 596
Chapter 20: Beyond Human-Designed Prompts with Generative Ideation 599
Part I: Defining generative ideation ....................................................................................... 600
Automated ideation architecture 601
Scope and limitations 602
Part II: Automating prompt design for generative image design ............................................. 602
ChatGPT/GPT-4 HTML presentation 603
ChatGPT with GPT-4 provides the text for the presentation 603
ChatGPT with GPT-4 provides a graph in HTML to illustrate the presentation 605
Llama 2 608
A brief introduction to Llama 2 608
Implementing Llama 2 with Hugging Face 609
Midjourney 615
Discord API for Midjourney 617
Microsoft Designer 622
Part III: Automated generative ideation with Stable Diffusion ................................................ 626
1. No prompt: Automated instruction for GPT-4 627
2. Generative AI (prompt generation) using ChatGPT with GPT-4 629
3. and 4. Generative AI with Stable Diffusion and displaying images 631
The future is yours! .............................................................................................................. 633
The future of development through VR-AI 633
The groundbreaking shift: Parallelization of development through the fusion of VR and AI 634
Opportunities and risks 637
Summary ............................................................................................................................. 637
Questions ............................................................................................................................. 638
References ........................................................................................................................... 638
Further reading .................................................................................................................... 639
Appendix A: Revolutionizing AI: The Power of Optimized
Time Complexity in Transformer Models 641
How constant time complexity O(1) of an operation changed our lives forever ........................ 641
O(1) attention conquers O(n) recurrent methods 642
Attention layer 642
Recurrent layer 643
The magic of the computational time complexity of an attention layer 644
Computational time complexity with a CPU 644
Computational time complexity with a GPU 647
Computational time complexity with a TPU 649
TPU-LLM 652
How one token sparked an AI revolution ............................................................................... 653
Appendix B: Answers to the Questions 657
Other Books You May Enjoy 677
Index 681
