Environment Variables

There are a number of environment variables that you can set to have even more granular control over some aspects of Venator, detailed below:

GOMAXPROCS

GOMAXPROCS (integer, optional): Overrides the maximum number of threads available to the program.

This is a very low-level tuning parameter you likely do not need to touch.

From the Golang documentation:

The GOMAXPROCS variable limits the number of operating system threads that can execute user-level Go code simultaneously. There is no limit to the number of threads that can be blocked in system calls on behalf of Go code; those do not count against the GOMAXPROCS limit.

If not provided, the value is determined via an appropriate default value from a combination of (source)

• the number of logical CPUs on the machine,
• the process’s CPU affinity mask,
• and, on Linux, the process’s average CPU throughput limit based on cgroup CPU quota, if any.

Venator uses this value a lot to determine the maximum number of concurrent threads that a routine can spawn for operations that may spawn an unbounded number of threads (such as global profile updates). In cases where this limit is explicitly checked for, as many threads as required will be created, but only up to N will be executed - the rest will be waiting on a semaphore.

Keep in mind not all operations that split into concurrent routines respect this limit, only those that expect to spawn a lot.

If you are on a single or even dual-core machine, you may wish to raise this. You can see the calculated value at runtime by running venatorctl debug, or by checking POST /_venator/v0/admin/metrics for max_threads.

VENATOR_CONFIG

VENATOR_CONFIG (non-empty string): The path where the configuration file for Venator can be found.

Overrides the path provided to venatorctl --config, if any.

VENATOR_TEST_DB_URI

VENATOR_TEST_DB_URI (non-empty string): The fully qualified postgresql:// connection URI for a test database.

Used for integration testing - set this if you are running the test suite with go test. Otherwise, database tests will be skipped. The database will be destroyed and recreated repeatedly!

VENATOR_ALLOW_MULTIPLE_SIGNING_KEYS

VENATOR_ALLOW_MULTIPLE_SIGNING_KEYS (literal 1): When value equals 1, permits having multiple active signing keys.

By default, when starting up, Venator will immediately invalidate all but one signing key if multiple active and valid ones are found. If, for whatever reason, you do not wish for this to happen, setting this env var will skip this process. This voids your warranty.

VENATOR_NO_CACHE_PREALLOC

VENATOR_NO_CACHE_PREALLOC (boolean): When enabled key-value in-memory caches will not be pre-allocated.

To reduce the number of memory allocations Venator has to do to cache new entries in its various key-value caches, it will pre-allocate all the space it expects to need for each cache at initialisation time. This has the benefit that cache operations will be faster as they will not need to rely on the Go runtime dynamically resizing the underlying storage. It has the downside, however, that Venator will appear to waste memory until the caches fill up.

If the increased performance cost at the benefit of decreased initial memory usage is something you need, the pre-allocation can be disabled by setting this environment variable. This is typically not necessary, however. Ideal cache sizes are constantly re-evaluated based on real-world data from the demo instance, and some caches can be manually adjusted.