We’re excited to invite you to join the TiCDC New Architecture Trial from February 13 to March 14, 2025. This is your chance to test the new architecture of TiCDC, the preferred incremental data synchronization tool in TiDB, and help shape its future!
Why Join the TiCDC New Architecture Trial?
TiCDC has been a crucial component of TiDB’s ecosystem, supporting disaster recovery, data integration, real-time analytics, and more. With the upcoming release of a new TiDB Version, TiCDC will introduce a new architecture designed to enhance performance, stability, and scalability.
As a beta tester, you’ll get to experience firsthand these exciting improvements and provide valuable feedback that will help us refine and optimize the system before the full release.
Participation Rewards
-
Exclusive Badge: Earn the TiCDC Pioneer badge to showcase your participation in this TiCDC New Architecture trial. This badge will be a symbol of your contribution to the TiDB community.
-
TiDB Backpack: Publish a TiCDC new architecture test article to receive a TiDB backpack. The article can be posted on any platform, but must include feedback from your experience.
-
Be Part of the Future: Play a key role in shaping the future of TiCDC with your feedback and suggestions.
-
Expand Your Expertise: familiarize yourself with the principles and usage of both the old and new TiCDC architectures through testing, and enhance your technical skills.
Advantages of the New TiCDC Architecture
The new TiCDC architecture brings significant improvements across several areas, making it a highly anticipated upgrade. Here’s what you can expect:
- Single-node Performance Improvement
-
A TiCDC node can support up to 500K tables.
-
A TiCDC node can handle up to 200MiB/s traffic in a wide-table scenario.
- Highly Horizontal scaleble
-
TiCDC is expected to scale out to more than 100 nodes.
-
It can support over 10K changefeeds.
-
One changefeed can manage millions of tables.
- Improved Stability
-
In high-write traffic scenarios, TiCDC maintains low replication latency and improved stability.
-
During TiCDC cluster scale-out, scale-in, and changefeed operations (e.g., creating or deleting changefeeds), the impact on unrelated changefeeds is minimized.
- Resource Efficiency and Cost Reduction
- In typical scenarios, TiCDC shows up to 10x improvement in CPU and memory efficiency, leading to significant cost reductions.
Additionally, the new architecture reduces code complexity, making it easier to accelerate the development of TiCDC. It’s also designed with future needs in mind, including support for Cloud-Native architectures and independent TiCDC deployment.
How to Participate?
Joining the TiCDC New Architecture Trial is simple! Here’s how you can get started:
1. Sign Up for the Beta Test:
Click on the registration link to sign up and secure your spot in the trial.
2. Requirements to Join:
To participate, you need to run TiDB v7.5.0 or higher. Ensure your TiDB cluster is up to date to start testing the new architecture.
3. Read the “Introduction to the New TiCDC Architecture”:
Before you start testing, make sure to read through the Introduction to the new architectural TiCDC to understand the key improvements and features.
4. Start Testing:
There are two ways to experience TiCDC.
- For users who want to test the new architecture, we recommend following the experience flow in the “Testing the New TiCDC Architecture” section.
- For users who have never used TiCDC before, the “15-Minute Quick Experience” will help you quickly understand TiCDC’s features.
Testing the New TiCDC Architecture: Official Experience Flow Based on TiUP Cluster
If you want to experience the new TiCDC architecture in a more formal environment, you can follow the process outlined here: Quick Start Guide.
15-Minute Quick Experience: Step-by-Step Guide Using TiUP Playground
If you’ve never used TiCDC or want to experience TiCDC from scratch and don’t have an existing TiDB cluster environment, you can follow the process below. Simply copy and paste the commands below to complete the entire experience!
Step 1:Start the upstream TiDB cluster
TICDC_NEWARCH=true tiup playground nightly --ticdc 1 --tiflash 0
Step 2:Start the downstream TiDB cluster
tiup playground nightly --tiflash 0 --db.port 4050
Step 3:Create a changefeed synchronization task
# Create a changefeed with the ID “demo”, syncing all tables with primary keys or unique non-null indexes from the upstream cluster to the downstream TiDB cluster
tiup cdc:nightly cli changefeed create --server=http://127.0.0.1:8300 --sink-uri="mysql://root@127.0.0.1:4050/" --changefeed-id="demo"
Step 4:At this point, the new architecture is configured. You can run any load on the upstream cluster, such as running the YCSB load
tiup bench ycsb load tidb -p tidb.instances="127.0.0.1:4000" -p recordcount=10000
Step 5:Connect to the downstream cluster and check the data synchronization status
mysql --host 127.0.0.1 --port 4050 -u root -e "select count(*) from test.usertable"
5. Where to Report Issues & Feedback:
As you test TiCDC’s new architecture, any issues or feedback should be reported on our official GitHub page:
Testing Directions
-
Simulate your normal business operations. If you encounter any bugs, please provide feedback here: ticdc/issues. We will address them as soon as possible.
-
Observe whether the delay lag is more stable than before, especially during peak business periods.
-
Monitoring if resource utilization (CPU, memory) has significantly decreased.
-
Test the scaling of the TiCDC cluster, or perform operations like creating, deleting, pausing, and resuming changefeeds. Observe the impact on changefeed lag, whether traffic remains stable during the process, and if there are any imbalances.
-
If you previously had to split the changefeed or even the cluster due to too many tables, frequent DDLs, or high traffic, try using only one cluster or a few changefeeds in the new architecture and see if they can be synchronized properly.
-
Evaluate how the new TiCDC architecture can improve your business operations.
What We Expect from Beta Tester
Your participation is invaluable, and while it’s not mandatory, we’d love your help with the following:
-
Provide Feedback: Share your insights and any issues you encounter while testing the new architecture by submitting them on the https://github.com/pingcap/ticdc/issues.
-
Write an Experience Blog: Write a blog or share your experience testing the new architecture to help the community understand its capabilities.
- You can publish it on any platform you prefer. Once published, share the blog link in the TiCDC Issues section on GitHub to let us know, and you’ll receive your reward.
