Getting Started with SaltStack Config: Working With Beacons

If you have been following this series, you should have an idea about SaltStack state files and Pillars. In this article, we’ll look at Beacons – a feature of SaltStack that allows us to monitor the events caused by the conditions of a machine.

Before diving into the material, let’s take a step back and understand what Beacons are and how they can help to manage your environment.

What are SaltStack Beacons?

SaltStack Beacons are a way to monitor events and conditions on a machine running a Salt Minion. For example, Beacons can be used to monitor disk space, memory usage, services and file changes. The charming part of this comes with another feature of SaltStack called Reactors. While the Beacon notifies the Salt Master of an event, the Reactor performs an action, such as reapplying a state to a target to restore the machine to its original state.

How do SaltStack Beacons Work?

SaltStack Beacons work by configuring the Salt Minion to emit events. These events are generated by Beacons. The basis of Beacons are Python scripts which are stored on the managed machine. Normally, you can find these Beacon scripts in /usr/lib/python3…./salt/beacons. The location might be different based on your installed version of python or your Linux distribution.

But how do these scripts work? If you take a closer look at these scripts, you will notice that they retrieve data from the underlying systems and convert them into measurable attributes that are compared to the thresholds and criteria in the Beacon configuration. If an event is generated, the information about it is returned to the master’s event bus.

Let’s take a closer look at one Beacon to understand its functionality, the Memory Usage Beacon.

The memusage Beacon monitors the current memory usage of your system and reports the percentage value back to the Salt Master when it exceeds the specified threshold. At this point, I mentioned two things: the threshold and the return value.

Let’s start with the return value. The memusage script uses a library that is included in the psutil package. You can play with psutil by installing the ‘psutil’ package on your own machine and reviewing the performance data.

The second part of the Beacon script is to compare the current value of memory usage with the threshold configured for the Beacon.

This threshold value comes from the Beacon configuration. The Beacon configuration can be created or appended in the following files and locations:

1. Place a beacon.conf file in /etc/salt/minion.d/
2. Append the /etc/salt/minion file
3. Add the Beacon configuration to a Pillar available for the Minion.

In all three cases, add a block that starts with Beacons:

beacons:
 memusage:
  - percent: 63%
  - disable_during_state_run: True

Of course, you can do all this directly on the master nodes, but since this series is about SaltStack Config, we will do the Beacon configuration via SaltStack Config UI. For this, we will use the file server of SaltStack Config, Jobs and States.

After deciding on a Beacon, we need to create its configuration. Since there are several ways to apply a Beacon, we first create the configuration. In this example, we will use the Beacon for memory usage. The configuration to be used could look like the example above.

Create a directory in your environment called Beacon, this is to keep the structure of your file server clean and create your Beacon states and conf-files in there.

Now that you know what your Beacon configuration should look like, we will use a state to apply that configuration to your Minion. I will show you two different ways to apply the configurations, so you can decide which one works best for you.

The first way is to copy the configuration from the SaltStack Config File Server to the target machine.

To accomplish this, we create a new state file and use the file.managed function.

install_beacon:
  file.managed:
    - name: /etc/salt/minion.d/beacon.conf
    - source: salt://beacon/files/beacon.conf
    - makedirs: True

If you decide to go this way, make sure that the state and configuration file is structured and formatted correctly.

The second option I want to show you does not require the creation of beacon.conf on the File Server. This time we will not use the file.managed function, instead we will use beacon.present: in our state file to apply the Beacon to your Minion.

For this alternative, create a Statefile like below:

memory_Beacon:
 beacon.present:
  - name: memusage
  - save: True
  - percent: 10%
  - disable_during_state_run: True

A complete overview of available Beacons, can be found here: https://docs.saltproject.io/en/latest/ref/beacons/all/index.html

The prework is done and your Beacon installation is prepared. To install it, we use a job and run it on a single Minion. Go to Jobs and create a job like the following example:

salt-call beacons.list

If your Beacon configuration was installed, you should see an output similar to the one below

In the example above, the Beacon would emit an event to the Salt Event Bus once the memory usage is more than 63%.

All the steps so far were to configure the Minions Beacon, but what does the Salt Master see when an event is being sent?

Run the following command on your Salt Master to see if an event notification is received.

salt-run state.event pretty=true

This will start the event runner and show you the results of the events received.

For demo purposes, I have decreased the threshold to 10%, which triggers the event.

Disable a Beacon

Now there are situations in every environment in which you have to do maintenance or for a certain reason you don’t want to monitor your systems. For this case it may be necessary to disable your Beacons.

As with the initial configuration of the Beacon, you can use a state file and apply it to your Minion via a job. Create a new state file like in the next example to disable a Beacon:

disable_beacon:
  beacon.disabled:
    - name: memusage

This will cause your Beacon to persist on your machine, but to be disabled.
If you run the command “salt-call beacons.list” again, you will see that the Beacon has been given another attribute: “enabled: false”

salt-call beacons.list

SaltStack Beacons are a powerful tool that can be used to monitor and respond to events.

More examples of how Beacons can be used are available in the official VMware documentation at https://docs.vmware.com/en/VMware-vRealize-Automation-SaltStack-Config/8.9/getting-started-saltstack-config/GUID-FDDE5251-3F3C-46C0-93B9-C54CBC0B0A6C.html

In the next article, we will look at Reactors and how you can monitor your Apache service status, for this we will use the Beacon example below:

apache_Beacon:
  beacon.present:
    - name: service
    - save: False
    - enabled: True
    - services:
        httpd:
          onchangeonly: True

Learn more about the power of SaltStack Beacons in combination with Reactors by reading our next blog post! Don’t miss out on the next post to level up your SaltStack game.