Local Worker
Every client has a local built-in worker. The local worker is used to runs jobs in two scenarios.
First, if no broker configuration value is specified, then the
client runs in standalone mode. In this mode, all jobs are executed by the
local worker.
Second, some jobs are considered local-only. These jobs must be run on the local machine because they utilize some resource that is only available locally. These jobs are always run on the local worker, even if the client is connected to a broker.
Currently, when a client is connected to a broker, it will only use the local worker for local-only jobs. This will change in the future so that the local worker is utilized even when the client is connected to a cluster.
Clients have the following configuration values to configure their local workers:
| Value | Type | Description | Default |
|---|---|---|---|
cache-size | string | target cache disk space usage | "1 GB" |
inline-limit | string | maximum amount of captured standard output and error | "1 MB" |
slots | number | job slots available | 1 per CPU |
cache-size
The cache-size configuration value
specifies a target size for the cache. Its default value is 1 GB. When the
cache consumes more than this amount of space, the worker will remove unused
cache entries until the size is below this value.
It's important to note that this isn't a hard limit, and the worker will go
above this amount in two cases. First, the worker always needs all of the
currently-executing jobs' layers in cache. Second, the worker currently first
downloads an artifact in its entirety, then adds it to the cache, then removes
old values if the cache has grown too large. In this scenario, the combined
size of the downloading artifact and the cache may exceed cache-size.
For these reasons, it's important to leave some wiggle room in the cache-size setting.
inline-limit
The inline-limit configuration
value specifies how many bytes of stdout or stderr will be captured from jobs.
Its default value is 1 MB. If stdout or stderr grows larger, the client
will be given inline-limit bytes
and told that the rest of the data was truncated.
In the future we will add support for the worker storing all of stdout and
stderr if they exceed inline-limit.
The client would then be able to download it "out of band".
slots
The slots configuration value specifies how many jobs the worker will run
concurrently. Its default value is the number of CPU cores on the machine. In
the future, we will add support for jobs consuming more than one slot.