Mastodon Instance Redis Tuning for Sidekiq: Memory Allocation Guide

Mastodon instances rely heavily on Redis as a high-speed cache and job queue for Sidekiq. When the Redis memory limit is set too low, Sidekiq jobs fail silently or are evicted, causing delayed or lost federated posts, notification failures, and timeline gaps. This guide explains how Redis memory allocation interacts with Sidekiq queues and provides concrete steps to tune the maxmemory setting for your Mastodon instance. By the end, you will know how to calculate a safe memory limit, apply the configuration, and verify that Sidekiq queues remain stable.

Why Redis Memory Matters for Sidekiq Queue Stability

Mastodon uses Redis for two main purposes: caching remote content such as user profiles and media metadata, and as the backend for Sidekiq job queues. Sidekiq pushes jobs for every federated action: posting a status, following a user, boosting a toot, and delivering activity to remote instances. Each job occupies a small amount of memory in Redis. When the total memory usage hits the maxmemory limit, Redis must evict keys according to its eviction policy. If the policy is set to noeviction, Redis returns errors to Sidekiq, and jobs are lost. If the policy is set to allkeys-lru or volatile-lru, Redis evicts the least recently used keys. This can include Sidekiq queue entries if the cache is full, leading to delayed or skipped job processing.

The default Redis configuration on many Linux distributions sets maxmemory to 0, which means no limit. On small instances with limited RAM, this can cause the system to swap or run out of memory entirely. On large instances, an unbounded Redis can consume all available RAM, leaving no room for the operating system or other services. A properly tuned maxmemory value prevents both scenarios by reserving enough headroom for Sidekiq queues while capping Redis cache growth.

How Sidekiq Uses Redis Memory

Sidekiq stores job payloads as JSON strings in Redis lists. Each job typically consumes 1 to 5 KB, depending on the size of the serialized arguments. For a Mastodon instance with 10,000 active users, Sidekiq may hold tens of thousands of jobs at any moment. Additionally, Sidekiq uses a separate Redis key for each queue, retry set, and scheduled job set. The sum of these keys can reach several hundred megabytes. The remaining Redis memory is used for caching remote objects, which can grow quickly on federated instances. Tuning must account for both the Sidekiq working set and the cache working set.

Steps to Tune Redis Memory for Sidekiq Queues

Follow these steps to calculate, apply, and verify a safe Redis memory limit for your Mastodon instance. All commands require root or sudo access on the Mastodon server.

1. Measure current Redis memory usage
   Run redis-cli INFO memory | grep -E 'used_memory_human|maxmemory_human' on the Mastodon server. Note the used_memory_human value. This shows how much memory Redis is currently using. If maxmemory_human shows 0, no limit is set.
2. Determine available system RAM
   Run free -h to see total RAM and available RAM. Reserve at least 1 GB for the operating system and other services. For a Mastodon instance running PostgreSQL and the web server, reserve 2 GB minimum. The remaining RAM can be allocated to Redis.
3. Calculate a safe maxmemory value
   Set maxmemory to 60 to 75 percent of total system RAM, minus the reserved amount. For example, on a server with 8 GB RAM and a 2 GB reservation: available for Redis = 6 GB. Use 4 GB as maxmemory to leave headroom. Write this value in megabytes: 4096 MB.
4. Edit the Redis configuration file
   Open /etc/redis/redis.conf with a text editor as root. Find the line # maxmemory . Uncomment it and set the value: maxmemory 4096mb. Ensure the unit is lowercase mb.
5. Set the eviction policy
   In the same file, find # maxmemory-policy noeviction. Change it to maxmemory-policy allkeys-lru. This policy evicts the least recently used keys across all Redis databases, keeping the most active Sidekiq jobs and cache entries.
6. Restart Redis
   Run sudo systemctl restart redis to apply the changes. Verify that Redis is running with sudo systemctl status redis.
7. Verify the new memory limit
   Run redis-cli INFO memory | grep -E 'used_memory_human|maxmemory_human' again. Confirm that maxmemory_human matches your configured value. Run redis-cli INFO stats | grep evicted_keys to see the eviction count. A low or zero eviction count indicates the limit is sufficient.
8. Monitor Sidekiq queue health
   Open the Sidekiq web interface at https://your-instance.example.com/sidekiq. Check the Queues tab for backlog growth. If queues grow faster than they are processed, consider increasing maxmemory or reducing the cache TTL in Mastodon settings.

Common Redis Memory Configuration Mistakes

Redis Not Restarting After Configuration Change

If Redis fails to restart after editing redis.conf, the most common cause is a syntax error in the configuration file. Run sudo redis-server /etc/redis/redis.conf --check-config to validate the file. Ensure the maxmemory value uses a valid unit: mb, gb, or plain bytes. Remove any trailing spaces.

Sidekiq Jobs Still Failing After Tuning

If Sidekiq continues to report job errors, check the Redis eviction count with redis-cli INFO stats | grep evicted_keys. A high eviction rate means the maxmemory is still too low. Increase the value by 25 percent and restart Redis. Alternatively, reduce the Redis cache TTL in Mastodon by setting CACHE_TTL = 600 in .env.production to expire cached remote objects faster.

Redis Using More Memory Than the Limit

Redis enforces the maxmemory limit asynchronously. During peak write bursts, memory may temporarily exceed the limit. This is normal and Redis will evict keys in the next maintenance cycle. If the overage persists, increase maxmemory by 10 percent.

Redis Memory Allocation Strategies: Fixed Limit vs Dynamic Allocation

Item	Fixed Limit (maxmemory)	Dynamic Allocation (no limit)
Memory control	Explicit ceiling prevents runaway usage	Redis uses all available RAM until OOM
Sidekiq reliability	Jobs are protected by LRU eviction policy	Jobs may be lost if system runs out of memory
Performance tuning	Requires monitoring and periodic adjustment	No configuration needed but risk of swap
Best for	Production instances with predictable traffic	Development or very small personal instances

The fixed limit approach is strongly recommended for any Mastodon instance serving more than one user. It provides predictable memory usage and prevents the entire server from becoming unstable. Dynamic allocation may work for single-user instances with ample RAM, but it risks OOM kills during federation spikes.

After tuning, you can now prevent Sidekiq job loss by setting a calculated maxmemory and allkeys-lru eviction policy in /etc/redis/redis.conf. Monitor used_memory and evicted_keys weekly to detect growth trends. For advanced monitoring, consider adding the redis-stat tool or Prometheus Redis exporter to your instance. Regularly review Mastodon cache TTL settings to keep the Redis working set aligned with your server resources.