Apply Salt State Every 15 Minutes

Automatically configure hundreds of computers while you sleep. To be more specific, schedule Salt state.apply to run every 15 minutes on slaves.

$ sudo salt '*' schedule.add tapply function='state.apply' minutes=15

This short article shows one way to do scheduled state.apply runs. I have not extensively used this method, but might move to it from an old one.

Start from a Working Salt Setup # ↑

This article start from a working Salt with some working idempotent configuration. If you're not there yet, start with Salt Quickstart – Salt Stack Master and Slave on Ubuntu Linux and Salt States – I Want My Computers Like This

Test that you can reach your slaves

$ sudo salt '*' cmd.run hostname
tmaster:
    tmaster

Test your idempotent configuration

$ sudo salt '*' state.apply --state-output=terse
tmaster:
  Name: /tmp/heippavaan - Function: file.managed - Result: Clean Started: - 12:19:21.091787 Duration: 20.79 m
  Name: apache2 - Function: pkg.installed - Result: Clean Started: - 12:19:22.523383 Duration: 476.905 ms
  // ..
Summary for tmaster
------------
Succeeded: 7
Failed:    0
------------
Total states run:     7
Total run time: 554.408 ms

List Scheduled Jobs # ↑

Let's see what tasks we have scheduled on slaves. If you want to remove all previous scheduled jobs, "sudo salt '*' schedule.purge", no undo.

$ sudo salt '*' schedule.list
tmaster:
    ----------
    schedule:
        ----------

No jobs yet.

Run state.apply Too Often - For Testing # ↑

Let's add a scheduled job. For testing purposes, we run this too often, twice a minute.

$ sudo salt '*' schedule.add applyTooOften function='state.apply' seconds=30

Here it is

$ sudo salt '*' schedule.list
tmaster:
    schedule:
      applyTooOften:
        enabled: true
        function: state.apply
        jid_include: true
        maxrunning: 1
        name: applyTooOften
        seconds: 30

Test # ↑

Let's test it by creating a new state, adding it to top file. But we will not run state.apply, we'll leave that to schedule.

On slave, let's check our test file is not there yet.

$ ls /tmp/tero
ls: cannot access '/tmp/tero': No such file or directory

Create a new state

$ sudo mkdir /srv/salt/tero/
$ sudoedit /srv/salt/tero/init.sls

# init.sls
/tmp/tero:
  file.managed

So it should create an empty file.

Let's add it to our top file, so that it will be included in state.apply without specifying state name.

# /srv/salt/top.sls:
base:
  '*':
    - tero # added this
    - heippa
    - apache
  'tmaster':
    - ssh

Don't run state.apply. You don't need to. The scheduler will do it.

Let's see if the empty file appered on slave:

$ ls /tmp/tero
/tmp/tero

We can see the file, so configuration was applied automatically.

We can see job result on master

$ sudo salt-run jobs.last_run

Apply Every 15 Minutes # ↑

Let's use a sensible setting of applying configuration every 15 minutes.

Let's remove our test job that runs twice a minute. If you have scheduled other jobs, you should just remove the applylTooOften job, but here we remove all scheduled jobs.

$ sudo salt '*' schedule.purge
.. Deleted job: applyTooOften from schedule. ..

$ sudo salt '*' schedule.add tapply function='state.apply' minutes=15
tmaster:
    ----------
    comment:
        Added job: tapply to schedule.
    result:
        True

Well done, now the configuration is automatically applied every 15 minutes. Salt is awake even when you sleep.

What next? If you have a huge network, you might want to add some splay (randomization) to times so that all slaves don't run state.apply at the same time.

Comments

Loading comments with JavaScript... You can read comments directly even without JavaScript.