Django With Redis Caching And Disaster Recovery

1.3 Django with Redis caching and Disaster Recovery

Overview

note

This section will guide you through setting up Redis caching in a Django application and outline strategies for disaster recovery to keep your data safe during system failures. We'll discuss how to ensure your application continues to operate smoothly, even when Redis is temporarily unavailable, by gracefully degrading to non-cached data retrieval.

Installation

1. Create a Custom Cache Backend

To maintain resilience, create a custom backend that handles Redis connectivity issues. This backend will use Redis when possible but will switch to a non-caching mode during outages.

Creating the Custom Backend:

1. Create a new Python file in your project, e.g., my_project/my_app/custom_redis_cache.py.
2. Extend the existing Redis backend with additional error handling:

from django.core.cache.backends.redis import RedisCache
from redis.exceptions import ConnectionError
import logging

logger = logging.getLogger(__name__)

class SafeRedisCache(RedisCache):
  def get(self, key, default=None, version=None):
    try:
      return super().get(key, default, version)
    except ConnectionError as e:
      logger.error(f"Redis connection error on GET operation: {e}")
      return default

  def set(self, key, value, timeout=None, version=None):
    try:
      super().set(key, value, timeout, version)
    except ConnectionError as e:
      logger.error(f"Redis connection error on SET operation: {e}")

2. Update Your Settings

Switch your caching backend in settings.py to use your robust custom backend. This helps minimize downtime by ignoring Redis when it's facing issues, allowing your site to operate, albeit more slowly.

"""
default caching with redis and this way is not recommended for production.
because when redis has issues, example: connection refused, site will be down
so switch to use `custom_redis_cache.py`
"""
# CACHES = {
#   "default": {
#     "BACKEND": "django.core.cache.backends.redis.RedisCache",
#     "LOCATION": "redis://localhost:6378/",
#     "KEY_PREFIX": "manga",
#     "TIMEOUT": 60 * 1,  # in seconds: 60 * 1 (1 minute)
#   }
# }

"""
with custom_redis_cache, site will be safe with minimal downtime
cause of ignoring redis if it has issue, so at that time site will be slow
but still working and we buy time to fix redis issue
"""
CACHES = {
  "default": {
    "BACKEND": "manga.custom_redis_cache.SafeRedisCache",
    "LOCATION": "redis://localhost:6378/",
    "KEY_PREFIX": "manga",
    "TIMEOUT": 60 * 1,  # 1 minute
  }
}

3. Adjust Your Views

With your new custom cache backend, views will handle Redis errors internally, so no changes are needed in most cases. However, implementing a fallback strategy in critical views can be prudent:

@cache_page(CACHE_TTL)
def index(request):
  try:
    # get all categories with cache
    items_category = cache.get('cache_items_category')
    if not items_category:
      items_category = Category.objects.filter(
        is_show_on_homepage=True,
        status=APP_VALUE_CATEGORY_CHOICE_STATUS_DEFAULT
      ).order_by('name')
      cache.set('cache_items_category', items_category, 20)  # 20 seconds expire
  except ConnectionError as e:
    print(f"error {e}")
    # get all categories without cache when redis issues
    items_category = Category.objects.filter(
      is_show_on_homepage=True,
      status=APP_VALUE_CATEGORY_CHOICE_STATUS_DEFAULT
    ).order_by('name')