Apply Salt State Every 15 Minutes

Automatically configure hundreds of computers while you sleep. To be more specific, schedule Salt state.apply to run every 15 minutes on slaves.

$ sudo salt '*' schedule.add tapply function='state.apply' minutes=15

This short article shows one way to do scheduled state.apply runs. I have not extensively used this method, but might move to it from an old one.

Start from a Working Salt Setup

This article start from a working Salt with some working idempotent configuration. If you're not there yet, start with Salt Quickstart – Salt Stack Master and Slave on Ubuntu Linux and Salt States – I Want My Computers Like This

Test that you can reach your slaves

$ sudo salt '*' cmd.run hostname
tmaster:
    tmaster

Test your idempotent configuration

$ sudo salt '*' state.apply --state-output=terse
tmaster:
  Name: /tmp/heippavaan - Function: file.managed - Result: Clean Started: - 12:19:21.091787 Duration: 20.79 m
  Name: apache2 - Function: pkg.installed - Result: Clean Started: - 12:19:22.523383 Duration: 476.905 ms
  // ..
Summary for tmaster
------------
Succeeded: 7
Failed:    0
------------
Total states run:     7
Total run time: 554.408 ms

List Scheduled Jobs

Let's see what tasks we have scheduled on slaves. If you want to remove all previous scheduled jobs, "sudo salt '*' schedule.purge", no undo.

$ sudo salt '*' schedule.list
tmaster:
    ----------
    schedule:
        ----------

No jobs yet.

Run state.apply Too Often - For Testing

Let's add a scheduled job. For testing purposes, we run this too often, twice a minute.

$ sudo salt '*' schedule.add applyTooOften function='state.apply' seconds=30

Here it is

$ sudo salt '*' schedule.list
tmaster:
    schedule:
      applyTooOften:
        enabled: true
        function: state.apply
        jid_include: true
        maxrunning: 1
        name: applyTooOften
        seconds: 30

Test

Let's test it by creating a new state, adding it to top file. But we will not run state.apply, we'll leave that to schedule.

On slave, let's check our test file is not there yet.

$ ls /tmp/tero
ls: cannot access '/tmp/tero': No such file or directory

Create a new state

$ sudo mkdir /srv/salt/tero/
$ sudoedit /srv/salt/tero/init.sls

# init.sls
/tmp/tero:
  file.managed

So it should create an empty file.

Let's add it to our top file, so that it will be included in state.apply without specifying state name.

# /srv/salt/top.sls:
base:
  '*':
    - tero # added this
    - heippa
    - apache
  'tmaster':
    - ssh

Don't run state.apply. You don't need to. The scheduler will do it.

Let's see if the empty file appered on slave:

$ ls /tmp/tero
/tmp/tero

We can see the file, so configuration was applied automatically.

We can see job result on master

$ sudo salt-run jobs.last_run

Apply Every 15 Minutes

Let's use a sensible setting of applying configuration every 15 minutes.

Let's remove our test job that runs twice a minute. If you have scheduled other jobs, you should just remove the applylTooOften job, but here we remove all scheduled jobs.

$ sudo salt '*' schedule.purge
.. Deleted job: applyTooOften from schedule. ..

$ sudo salt '*' schedule.add tapply function='state.apply' minutes=15
tmaster:
    ----------
    comment:
        Added job: tapply to schedule.
    result:
        True

Well done, now the configuration is automatically applied every 15 minutes. Salt is awake even when you sleep.

What next? If you have a huge network, you might want to add some splay (randomization) to times so that all slaves don't run state.apply at the same time.