Oracle Cloud Infrastructure (OCI) GoldenGate: Real-world Examples / Облачная инфраструктура Oracle Golden Gate: Примеры из реального мира
Год издания: 2024
Автор: Kumar Ravi and others / Кумар Рави и другие
Издательство: Apress Media
ISBN: 979-8-8688-0303-1
Язык: Английский
Формат: PDF, EPUB
Качество: Издательский макет или текст (eBook)
Интерактивное оглавление: Да
Количество страниц: 243
Описание: This book focuses on the utilization of GoldenGate Services (GGS) in conjunction with a microservices architecture on the Oracle cloud (OCI), primarily for data migration and integration across various data sources and targets.
The book begins with a practical example of utilizing GGS on a Marketplace VM, progressively advancing to in-depth discussions on implementing GoldenGate as a Service on OCI. The book offers illustrative guides for data replication between RDBMSs (such as Oracle, Postgres, and big data targets such as Kafka). Additionally, it explores monitoring techniques using Enterprise Manager and Grafana dashboards. A comparative analysis is presented between traditional VM-based GoldenGate installations and the OCI service model. Special attention is given to Zero Downtime Migration (ZDM) and leveraging GGS for database migration from on-premises to OCI. Some chapters address multi-cloud replication using OCI GGS and include real-life case studies.
By the end of this book you will have gained comprehensive insights into the architectural design of GoldenGate Services and will be adept at replicating data using GGS, enabling you to replicate setups in your own environments.
What You Will Learn
Set up GoldenGate Services for high availability (HA), disaster recovery (DR), migration of data to cloud, and moving data into the data lake or lakehouse
Perform logical migration of data to the cloud using the ZDM tool (ZDM uses GoldenGate internally).
Replicate data to big data targets
Monitor GGS using Enterprise Manager and Grafana
Replicate data in a multi-cloud environment
Who This Book Is For
Oracle database administrators who want to replicate data or use Oracle GoldenGate Services for migration and setup of high availability (HA) and disaster recovery (DR); and data engineers who want tobuild the data warehouse, data lake, data lakehouse to push data in near real-time.
Эта книга посвящена использованию сервисов GoldenGate (GGS) в сочетании с архитектурой микросервисов в Oracle cloud (OCI), в первую очередь для миграции и интеграции данных из различных источников и целевых объектов.
Книга начинается с практического примера использования GGS на виртуальной машине Marketplace, постепенно переходя к подробному обсуждению внедрения GoldenGate как сервиса в OCI. В книге представлены иллюстративные руководства по репликации данных между СУБД (такими как Oracle, Postgres и объектами обработки больших данных, такими как Kafka). Кроме того, в ней рассматриваются методы мониторинга с использованием информационных панелей Enterprise Manager и Grafana. Представлен сравнительный анализ традиционных установок GoldenGate на базе виртуальной машины и модели обслуживания OCI. Особое внимание уделяется миграции с нулевым временем простоя (ZDM) и использованию GGS для переноса баз данных из локальной среды в OCI. В некоторых главах рассматривается репликация в нескольких облаках с использованием OCI GGS и приводятся примеры из реальной жизни.
К концу прочтения этой книги вы получите исчерпывающее представление об архитектурном дизайне сервисов GoldenGate и научитесь реплицировать данные с помощью GGS, что позволит вам воспроизводить настройки в ваших собственных средах.
Что вы узнаете
Настройте сервисы GoldenGate для обеспечения высокой доступности (HA), аварийного восстановления (DR), миграции данных в облако и переноса данных в data lake или lakehouse
Выполните логическую миграцию данных в облако с помощью инструмента ZDM (ZDM использует GoldenGate внутри компании).
Реплицируйте данные для целей обработки больших объемов данных
Контролируйте GGS с помощью Enterprise Manager и Grafana
Реплицируйте данные в мультиоблачной среде
Для кого предназначена эта книга
Администраторы баз данных Oracle, которые хотят реплицировать данные или использовать сервисы Oracle GoldenGate для миграции и настройки высокой доступности (HA) и аварийного восстановления (DR), а также инженеры по обработке данных, которые хотят создать хранилище данных, data lake, data lakehouse для передачи данных практически в режиме реального времени.
Примеры страниц (скриншоты)
Оглавление
About the Authors ......................................................................................xi
About the Technical Reviewers .....................................................................xv
Acknowledgments ......................................................................................xvii
Foreword ..................................................................................................xix
Chapter 3: Setting Up Oracle GoldenGate on the OCI Marketplace ....................15
Introduction to Oracle GoldenGate on the OCI Marketplace ..............................16
Preparing the OCI Environment for GoldenGate ..............................................17
How to Access the OCI Marketplace ..............................................................18
Launching Oracle GoldenGate from the OCI Marketplace ..................................20
Configuring the Oracle GoldenGate Marketplace ..............................................30
Add New Extract .........................................................................................38
Creating a Replicat ......................................................................................40
Testing the DML Replication ..........................................................................44
Using CLI AdminClient for Microservices Architecture ........................................47
Managing and Monitoring Marketplace GoldenGate ...........................................47
Summary ...................................................................................................48
Chapter 4: Setting Up OCI Oracle GoldenGate Service for RDBMS ......................51
Introduction to Oracle GoldenGate PaaS Service for RDBMS ..............................52
Provisioning Oracle GoldenGate Service for RDBMS ..........................................54
Create Deployment ......................................................................................54
Create Connection .......................................................................................58
Create Source Connection On-Premises Oracle Database ...................................58
Create Target Connection OCI Autonomous Database ........................................61
Accessing Oracle GoldenGate Service for RDBMS ..............................................64
Testing Source and Target DB Connection ........................................................66
Set Up and Configure Source Extract ..............................................................67
Set Up and Configure Target Replicat ..............................................................69
Summary ....................................................................................................72
Chapter 5: Setting Up OCI Oracle GoldenGate Service for BIGDATA ....................73
Introduction to Oracle GoldenGate Service for BIGDATA Targets .........................74
Provisioning Oracle GoldenGate Service for BIGDATA Targets ..............................76
Create Deployment ........................................................................................76
Create Connection .........................................................................................80
Create BIGDATA Target Kafka Connection ..........................................................80
Accessing Oracle GoldenGate Service of BIGDATA ...............................................84
Set Up and Configure Target Replicat ..................................................................85
Summary .........................................................................................................89
Chapter 6: Managing and Monitoring Oracle GoldenGate on OCI ...............................91
Monitoring Features ............................................................................................92
Notification Features ...........................................................................................92
Oracle Cloud Infrastructure (OCI) Observability and Management Services ..................93
What Is OCI Observability and Management Services? ..............................................93
Create Topic ........................................................................................................94
Create Subscription ............................................................................................95
Create Service Metrics and Alarms .....................................................................97
Custom Scripting to Alert via Email Notifications .............................................104
Using Grafana for Trend Dashboards and Alerts ...............................................106
Connecting OCI GoldenGate to Grafana .........................................................107
Summary ..................................................................................................112
Chapter 7: Comparing OCI GoldenGate Service with Cloud
Marketplace ...............................................................................................113
Use Cases Between OCI GoldenGate Services and Cloud Marketplace
GoldenGate ...............................................................................................115
Summary ..................................................................................................117
Chapter 8: Migration of Databases with Near-Zero Downtime ............................119
Install ZDM Software on OCI Linux Compute VM .............................................120
Provision the Compute VM ...........................................................................121
ZDM Software Installation ...........................................................................122
ZDM – Logical Online Migration Using GoldenGate ...........................................125
Setup Required ...........................................................................................125
Provision VM for Oracle GoldenGate – Database Migrations ...............................125
SSH Key Exchange for Passwordless Connectivity ...........................................126
Set Up the Source Database ........................................................................126
Set Up the GoldenGate Host ........................................................................128
Add Hostnames – ZDM HOST ......................................................................129
Create a ZDM Response File ........................................................................130
Testing the Whole Configuration ..................................................................132
Validate the Job Status ...............................................................................133
Start the Migration Process .........................................................................134
Validate the Job Status of Actual Migration ....................................................135
Complete the Migration Process ...................................................................137
Summary .................................................................................................138
Chapter 9: Troubleshooting Oracle GoldenGate on OCI ...................................139
Classify the Common Encountered Issues with GoldenGate .............................140
Diagnosing Connectivity Issues Between the Source and the Target ..................140
Performance Bottleneck Identification .............................................................142
High Transaction Volumes .................................................................................144
Data Conflicts and Consistency Issues .............................................................145
Logdump in OCI GGS .........................................................................................149
How to Use Logdump for OCI GGS? .............................................................149
OCI GoldenGate Diagnostics ggserr.log Rpt Files Log Files .........................151
Summary ..........................................................................................................154
Chapter 10: On-Premises Database Migration to OCI .......................155
Setup and Provisioning of OCI GoldenGate Service ..........................................155
Configure GoldenGate Extract to Capture from On-Premises ...........................156
Prepare the Source Database ...........................................................................156
1. Enable Minimum Supplemental Logging at the Database Level .............156
2. Create a Connection from OCI GoldenGate to the On-Premises
Database .................................................................................................159
3. Enable Minimum Supplemental Logging on the Object Level .................161
4. Configure the Extracts to Capture On-Premises
Changes .......................162
Configure a GoldenGate Replicat to Apply It to the OCI Database .....................165
1. Enable_Goldengate_Replication Parameter ............................................166
2. Create User and Tablespaces ..................................................................166
3. Create a Connection from OCI GoldenGate to OCI DBaaS ........................167
4. Configure the Replicat to Apply It to the Target OCI DBaaS Database .....169
Initial Data Load ................................................................................................173
1. Oracle Data Pump (EXPDP/IMPDP) ..........................................................173
2. RMAN Backup and Restore .....................................................................175
GoldenGate Initial Load Method ........................................................................177
Initial Load Extract .......................................................................................177
Initial Load Replicat .....................................................................................179
Point in Time to Start GoldenGate Replication ................................................181
Monitoring the OCI Golden Service Process ....................................................181
OCI Observability and Management Method ...................................................182
Oracle Enterprise Manager ..........................................................................182
Custom Scripts to Monitor OCI GG from the Compute VM ................................183
Grafana Monitoring .....................................................................................183
Cut Over the Replication from On-Premises .....................................................183
Synchronization and Data Catch-Up ...............................................................183
Post-Migration Tasks ...................................................................................185
Summary ..................................................................................................186
Chapter 11: Replicat Data Across Multicloud Using OCI
GoldenGate ...............................................................................................187
Architectural Consideration ..........................................................................189
Data Transfer Implications ...........................................................................189
Network Latency Implications .......................................................................190
Architectural Factors to Address These Issues .................................................190
Configuring OCI GoldenGate Services from OCI to AWS ....................................191
Prerequisites ...............................................................................................192
Create OCI GoldenGate Connections for OCI Database and AWS RDS .................194
Create a Connection to AWS RDS ...................................................................196
Create an Extract ........................................................................................198
Create a Replicat .........................................................................................201
Validating Transactions with OCI GoldenGate Services ......................................204
Understanding Transaction Validation ..............................................................204
Summary ....................................................................................................207
Chapter 12: Real-World Examples of Oracle GoldenGate on OCI ..........................209
Case Study 1: Implementing Active-Active
Database Replication on OCI
GoldenGate Services ......................................................................................210
Context ........................................................................................................210
Challenges ....................................................................................................211
Implementing Active-Active Replication with Oracle GoldenGate on OCI ................211
Gradual Transition to OCI ................................................................................211
Finalize Migration and Decommission On-Premises Setup ....................................212
Outcome ......................................................................................................212
Case Study 2: Real-Time Data Ingestion to BIGDATA Using OCI
GoldenGate Services ......................................................................................213
Context ........................................................................................................213
Solution .......................................................................................................214
Outcome ......................................................................................................214
Case Study 3: Fraud Detection with Streaming Analytics Using OCI
GoldenGate Services ......................................................................................214
Context ........................................................................................................214
Solution .......................................................................................................215
Key Components Using GoldenGate Fraud Detection Architecture .........................215
Benefits .......................................................................................................216
Outcome ......................................................................................................216
Case Study 4: SaaS Application Data into Enterprise ..........................................217
Context ........................................................................................................217
Solution .......................................................................................................218
Outcome ......................................................................................................218
Case Study 5: Disaster Recovery Solution with Oracle GoldenGate
Services on OCI ............................................................................................219
Context ........................................................................................................219
Challenges ...................................................................................................220
Solution .......................................................................................................220
Outcome ......................................................................................................221
Index ..........................................................................................................223
