Caches

caches: Controls the sizes and lifetimes of some runtime caches.

Venator caches some data in memory to avoid excessive round-trips to the database, especially when fetching that data may end up being expensive (e.g. fetching a lot of events in rapid succession). Generally, the size of these internal caches scales with your available resources (or, more accurately, every cache defaults to 4096*N, where N is the number of logical processors).

There are no magic numbers, if you need to tune your caches, you should do so with trial and error. What works best is different for each deployment.

Each cache map has two keys, max_entries, and max_ttl. max_entries (integer, optional) controls how many entries can be in the cache before old ones start being evicted. By default, it is 4096*N, where N is the number of logical CPUs. Set this to zero to disable limiting the size of caches (not recommended). max_ttl controls how long entries are allowed to remain in the cache before they are evicted. Set to zero to disable TTL eviction.

Warning

TTL-based cache evictions are checked on each cache operation (both read and write), which means they are inherently more computationally expensive than simple size-based limits.

On the other hand, size-based eviction is only evaluated on write operation, making them generally cheaper, but may result in Venator holding on to memory purely for “stale” cache entries.

It is recommended you leave caches as their default values unless you are encountering memory constraint issues.

Examples:

caches:
  events:
    max_entries: 8192
    max_ttl: 5m

You can monitor the cache size and hit/insert/eviction rate via /metrics.

accounts

accounts: Controls the accounts cache.

There are two caches for accounts which enable faster account lookups. This cache will always be hot, as accounts are looked up for each incoming request. As one of these caches is an access token to Account mapping (so that Bearer tokens can be instantly related to an account), having values that do not match your devices cache may cause weird behaviour.

See caches for more details.

Example:

caches:
  accounts:
    max_entries: 4096

account_data

account_data: Controls the account data cache.

Some account data types are frequently looked up, both by the server and the client, such as the ignored users list. In order to avoid running to the database every time one of these entries are looked up, they can be cached in-memory locally instead, which can significantly increase lookup speeds.

The account data cache is in a “flat” configuration, not scoped to per-user or per-type etc. If you only allow 100 entries, there can only be 100 rows cached.

See caches for more details.

caches:
  account_data:
    max_entries: 4096

devices

devices: Controls the devices cache.

There are three caches for devices which allow for faster device lookups for incoming requests. Note that the devices cache will be very hot when end-to-end encryption workloads are involved, so you should avoid lowering this one unless you know for sure you won’t be dealing with end-to-end encryption.

The default values for this cache are likely excessive when federation is not involved.

See caches for more details.

Example:

caches:
  devices:
    max_entries: 4096

events

events: Controls the event cache.

The event cache is a key-value map of {event_id: event_data}. Each value may be up to 64KiB. Generally you want quite a large event cache, and this is the first thing that will be hit during operations that involve events (so, most of them), for example: state resolution, fetching events, client sync loops, message pagination.

See caches for more details.

Example:

caches:
  events:
    max_entries: 8192