🐈 Task: Add Redis-Based Caching for Model API Results

[Abellegese]

Description

Currently, our Erilia does not implement caching for API results produced by Ersilia models. As a result, running models frequently creates redundant computations and unnecessary resource expenditure, which is particularly challenging for users operating in low-resource environments or with resource-intensive models.

To address this, we aim to integrate a Redis-based caching mechanism. Redis is a high-performance, lightweight, and widely-used in-memory database suitable for caching, especially given its compatibility with Docker. This new feature will allow results to be cached during each session, significantly reducing resource usage and improving user experience.

Implementation Design

• Redis Class Design:
  1. Async Implementation: The Redis client will use an asynchronous interface for non-blocking operations to optimize performance, especially during high-concurrency scenarios.
  2. Singleton Pattern: A singleton ensures only one instance of the Redis client exists during runtime, avoiding resource waste and ensuring consistent access to the Redis cache.
  3. Thread-Safe: To safely handle multiple threads accessing the Redis client, thread safety will be prioritized, ensuring no unexpected behaviors occur.
  4. Connection Pooling: Connection pooling improves performance by reusing existing connections instead of creating new ones for each request, reducing overhead and latency.
• High-level Mechanism: Each smiles will have a corresponding key that is perfect for caching. In each key the model id will be appended to cache smiles computation for each model. This will be implemented in the post function in both standard and conventional run. When we post first each smiles will be checked in the redis cache using redis high performance query function called mget. For the cached smiles their value will be returned and for non-cached smile, request will be made and their value will be cached for next time.

Why This Is Needed

Caching can significantly enhance performance by reducing duplicate computations. It also provides the following advantages:

• Improves speed for repeated API requests for the same inputs.
• Reduces server load, optimizing resources for intensive models.
• Enhances user experience, especially for users in constrained computing environments.

Tasks

• Fetch and configure the Redis Docker image for development and testing.
• Create the Redis client class that adheres to async, singleton, thread-safe, and connection pooling principles.
• Integrate the Redis client into the Standard API and Conventional Run to handle caching for model results.
• Add appropriate unit tests to ensure caching works as expected.
• Update the API documentation to include caching behavior and configuration settings.
• Create a troubleshooting section in the documentation for issues like Redis connection failure or unexpected cache behaviors.
• Ensure the application gracefully handles cases where Redis is unavailable.

Objective(s)

No response

Documentation

No response

[miquelduranfrigola]

Thanks @Abellegese - let's make sure this does not overlap with the GDI work (i.e. inference store in AWS). This relates to local caching.

[Abellegese]

Okay @miquelduranfrigola , but I still don't have a clear view of their thier inner mechanism for their caching for me not to duplicate. Maybe we need time for this as well.

[miquelduranfrigola]

Sure, of course. Let's plan this well.