I am wondering if it’s advised or even possible to use tikv (and/or tidis) in a tidb cluster; we have a tidb cluster but want to replace redis now (simpler ops); tikv instances are running clustered anyway so we would like to use them for the redis loads as well. Is that possible / advised? If possible and an ok idea: any way to make sure it doesn’t interfere with tidb?
It’s not recommended to directly use TiKV or TiDB instances within your existing TiDB cluster for Redis workloads. While TiKV is a distributed key-value storage engine, it’s specifically designed and optimized for the TiDB ecosystem and its particular requirements. Using it for Redis workloads could potentially cause performance issues, introduce compatibility problems, or even compromise the stability of your TiDB cluster.
Here’s why:
- Data Model and Access Patterns: TiKV is designed for transactional workloads and the complex SQL queries that TiDB handles. It uses a different data model and access patterns than Redis, which is optimized for fast key-value lookups and caching. Directly using TiKV for Redis workloads might lead to slower performance and unexpected behaviors.
- Compatibility: TiDB and TiKV are tightly integrated and rely on specific protocols and interactions. Using them for a different purpose might lead to compatibility issues, data corruption, or even data loss.
- Cluster Management: TiDB and TiKV are managed as a cluster, with specific requirements for resource allocation and scaling. Adding Redis workloads to the mix could complicate management and potentially negatively affect your cluster’s overall performance.
Instead of directly integrating TiKV or TiDB for Redis, consider using a separate Redis cluster or explore alternative solutions within the TiDB ecosystem, such as:
- TiDB-based Caching: Explore TiDB’s built-in caching capabilities for frequently accessed data. This might be a suitable alternative if your Redis use case involves caching.
- Separate Redis Deployment: Deploy a dedicated Redis cluster separate from your TiDB cluster. This ensures a dedicated environment for Redis workloads and avoids potential conflicts.
- Other TiDB Ecosystem Options: Research other components within the TiDB ecosystem that might better suit your Redis replacement needs.
Remember to carefully assess your use case and requirements before choosing a solution. Consulting with PingCAP support or the TiDB community can provide valuable insights and guidance.
1 Like
How does TiDB-based Caching: work? Because what I can find only relates to small config tables; we are looking at larger caches (not enormous but few 100mb).
TiDB offers a feature called “Cached Tables” for improving read performance on frequently accessed but rarely updated small hotspot tables. This feature involves loading the entire table data into the TiDB server’s memory, eliminating the need to access TiKV and speeding up read operations. However, this feature is currently optimized for smaller tables, with a size limit of 64 MB.
Therefore, for larger caches in the range of a few hundred megabytes, directly using TiDB’s “Cached Tables” feature is not recommended due to its size limitations.
Currently, TiDB does not provide a built-in solution for caching larger tables exceeding the 64 MB limit. You would need to explore alternative caching mechanisms or consider architectural changes to optimize performance for your specific use case.
How about a separate tikv/tidis cluster?
You are asking about the feasibility of deploying a separate TiKV/TiDIS cluster for Redis workloads. I understand you’re interested in:
- Compatibility: How well would this separate cluster integrate with your existing TiDB cluster?
- Performance: What impact would this have on both clusters?
- Management: Would there be additional complexities in managing these two clusters?
While the context does not contain information on TiDIS, based on my knowledge of TiDB:
- TiKV is designed for key-value storage and is used by TiDB as its primary storage engine. It is not directly intended for Redis use cases. Redis has its own unique features like high-performance in-memory data storage, which TiKV does not provide natively.
- Deploying a separate TiKV cluster for Redis might not be the most efficient approach due to the mismatch in functionalities and optimizations between TiKV and Redis.
- Integrating a separate TiKV cluster with your existing TiDB cluster would require careful consideration of data consistency and network communication. It might introduce complexities in managing data replication, synchronization, and potential performance bottlenecks.
Alternatives to consider:
- Using TiDB’s built-in support for key-value pairs: TiDB supports key-value data storage through its built-in functionalities, which might be suitable for certain Redis-like use cases.
- Deploying a dedicated Redis cluster: This is often the most efficient approach for managing Redis workloads, as it provides dedicated resources and optimizations for Redis functionality.
Recommendation:
For your specific Redis use case, I recommend consulting with a PingCAP expert or reviewing the TiDB documentation for more specific guidance on using TiDB for Redis-like workloads or on how to deploy a dedicated Redis cluster.
TiDB Cluster Management FAQs: Can both TiDB data and RawKV data be stored in the same TiKV cluster?