RAG with Python Cookbook / Книга рецептов RAG с Python
Год издания: 2026
Автор: Polzer Dominik / Польцер Доминик
Издательство: O’Reilly Media, Inc.
ISBN: 979-8-341-60056-0
Язык: Английский
Формат: PDF/EPUB
Качество: Издательский макет или текст (eBook)
Интерактивное оглавление: Да
Количество страниц: 378
Описание: As businesses race to unlock the full potential of large language models (LLMs), a critical challenge has emerged: How do you connect these tools to real-time, external data to solve real-world problems? Retrieval-augmented generation (RAG) is the answer. By combining LLMs with information retrieval, RAG empowers you to build everything from intelligent chatbots to autonomous, task-solving agents.
Packed with over 70 practical recipes, this go-to guide tackles a wide range of GenAI applications through structured hands-on learning. Author Dominik Polzer provides the tools you need to design, implement, and optimize RAG systems for your unique use cases. Whether you're working with simple data retrieval or designing cutting-edge autonomous agents, this cookbook will help you stay ahead of the curve.
Learn core RAG components including embedding, retrieval, and generation techniques
Understand advanced workflows like semantic-aware chunking and multi-query prompting
Build custom solutions such as chatbots and autonomous agents for specific data challenges
Continuously evaluate and optimize systems for accuracy, relevance, and performance
В то время как компании стремятся раскрыть весь потенциал больших языковых моделей (LLM), возникла серьезная проблема: как подключить эти инструменты к внешним данным в режиме реального времени для решения реальных задач? Поисково-расширенная генерация (RAG) - вот ответ. Сочетая LLMS с поиском информации, RAG позволяет создавать все - от интеллектуальных чат-ботов до автономных агентов, решающих задачи.
В этом практическом руководстве, содержащем более 70 рецептов, рассказывается о широком спектре приложений GenAI благодаря структурированному практическому обучению. Автор Доминик Польцер предоставляет инструменты, необходимые для проектирования, внедрения и оптимизации систем RAG для ваших уникальных случаев использования. Независимо от того, работаете ли вы с простым поиском данных или разрабатываете передовые автономные агенты, эта кулинарная книга поможет вам всегда быть на шаг впереди.
Изучите основные компоненты RAG, включая методы встраивания, поиска и генерации данных
Разберитесь с передовыми рабочими процессами, такими как сегментирование с учетом семантики и запрос нескольких запросов.
Создавайте индивидуальные решения, такие как чат-боты и автономные агенты, для решения конкретных задач с данными
Постоянно оценивайте и оптимизируйте системы с точки зрения точности, актуальности и производительности
Примеры страниц (скриншоты)
Оглавление
Preface vii
1. Getting Started with RAG 1
1.1 Identifying High-Value RAG Use Cases for Your Organization 3
1.2 Choosing Your IDE and Coding Agent Setup 6
1.3 Getting Started with Jupyter Notebooks in VS Code 8
1.4 Storing Secrets and API Keys with .env Files 10
1.5 Building Your First RAG App 13
1.6 Choosing the Frameworks and Libraries for Your RAG Applications 18
1.7 Running the Code Examples in the Book Repository 21
2. Foundation Models 23
2.1 Defining a Suitable Prompt Template 24
2.2 Selecting the Right Language Model for Your Task 26
2.3 Generating Content with the OpenAI API 29
2.4 Generating Content with Google’s Gemini Models 33
2.5 Generating Content with the Anthropic API 35
2.6 Running Open Source Models Locally with Ollama 37
2.7 Creating Structured Outputs with the OpenAI SDK and Pydantic 41
3. Loading Data 45
3.1 Loading Word Files in Python 47
3.2 Loading PDF Files 50
3.3 Loading and Handling Tabular Data from Excel and CSV Files 52
3.4 Loading Structured Data from a PostgreSQL Database 57
3.5 Loading Audio Files via Speech-to-Text Models 59
3.6 Extracting Text from Images and PDFs via Tesseract OCR 61
3.7 Extracting Text from Images via Multimodal Models 65
3.8 Generating Text Description for Images via Multimodal Models 68
3.9 Generating Text Summaries for Embedded Tables via Multimodal Models 70
3.10 Parsing PDFs with Multimodal Content 73
3.11 Loading Videos via Speech-to-Text and Multimodal Models 77
4. Data Preparation 83
4.1 Adding Metadata to Enable Metadata Filtering 84
4.2 Enhancing Data Quality by Replacing Abbreviations and Technical Terms 90
4.3 Improving Search Accuracy by Creating Hypothetical Questions for Text Chunks 95
4.4 Splitting Documents via Character Splitting 99
4.5 Splitting Documents with Recursive Text Splitters 101
4.6 Chunking Documents with Document-Aware Splitting 105
4.7 Splitting Text with Semantic-Aware Chunkers 108
4.8 Splitting Text with Agentic Chunkers 111
5. Embeddings 117
5.1 Mapping the Linguistic Meaning of Text Chunks to a Numerical Representation 118
5.2 Visualizing Semantic Relationships Between Text Chunks via Dimensionality Reduction Techniques 123
5.3 Calculating the Distance Between Embeddings 127
5.4 Choosing the Right Embedding Model 131
5.5 Generating Embeddings for Images and Text with CLIP 133
5.6 Performing Text Classification with Embeddings 137
5.7 Improving Search Results with a Hybrid Search Approach 141
6. Vector Databases and Similarity Searches 147
6.1 Choosing the Right Vector Database 148
6.2 Storing and Searching Embeddings with FAISS 151
6.3 Storing and Working with Embeddings in a Chroma Vector Database 155
6.4 Storing Embeddings in PostgreSQL with the pgvector Extension 159
6.5 Performing Similarity Search in PostgreSQL 164
6.6 Accelerating Vector Searches in PostgreSQL with Indexing Techniques 167
6.7 Combining Keyword and Similarity Search to Improve Retrieval Accuracy with PostgreSQL 173
7. Retrieval 177
7.1 Optimizing Query Results via Metadata Filtering in PostgreSQL 180
7.2 Enhancing Retrieval Accuracy with HyDE 184
7.3 Improving Search Results with Multiquery Retrieval 188
7.4 Addressing Complex Requests by Designing a Query Routing System 192
7.5 Enhancing Retrieved Documents by Designing an Auto-Merging Retriever 197
7.6 Retrieving More Complete Text Chunks with a Sentence Window Retriever 200
7.7 Improving Retrieval Relevancy with Reranking Methods 203
7.8 Decomposing Complex Queries into Multiple Subqueries 206
8. Agentic RAG 211
8.1 Designing a Custom Tool in Python 215
8.2 Using Workflow Patterns in Multiagent Systems 216
8.3 Choosing an Agentic Framework 221
8.4 Building an Agentic System via Function Calling 224
8.5 Accelerating Agents with asyncio 231
8.6 Building a Sales Negotiation Agent with OpenAI’s Agents SDK and Chroma 236
8.7 Enriching Your Agent’s Capabilities with MCP Tools 245
8.8 Building an Agentic System with LangGraph 250
9. Graph RAG 261
9.1 Creating Your First Neo4j Knowledge Graph and Feeding It with Text from Documents 263
9.2 Extending the Knowledge Graph with Structured Data 271
9.3 Building Your First Cypher Query 275
9.4 Enabling Semantic Search on a Neo4j Knowledge Graph 278
9.5 Optimizing the Knowledge Graph for RAG Systems 281
10. Evaluating RAG Systems 285
10.1 Choosing the Right Evaluation Metrics for RAG Systems 289
10.2 Evaluating RAG Systems by Humans 293
10.3 Creating Synthetic Data for Automated Testing 296
10.4 Evaluating the Retriever Step by Calculating Context Precision@k 301
10.5 Evaluating Faithfulness During Generation with LLM-as-a-Judge 308
10.6 Evaluating the Response Relevancy of Your RAG System 316
11. RAG Web Apps323
11.1 Building Your First Streamlit App 324
11.2 Building a Chatbot App with Streamlit 326
11.3 Adding PDF Analyzer Functionality to Your Chatbot 336
11.4 Connecting Your RAG App to a SQL Database 341
11.5 Deploying Your Streamlit App with Docker and AWS 347
Index 351
