Black Box Plugin

Purpose

The purpose of this plugin is to provide a "Black Box" recorder for the Ardexa edge device.

Usage

The Ardexa Black Box recorder, as the name suggests, records all relevant Ardexa edge environmental factors, to the Ardexa cloud, at regular intervals. Normally, the ArdexaLinux image contains all the necessary modules to manage the black box recorder. This plugin is an update to the ArdexaLinux Black Box module. As of Version 1.6.5 or greater, data will be sent to the cloud and be available in the ANALYSIS section of the Ardexa cloud. This plugin is scheduled to run every 5 minutes by the native Linux Cron subsystem. The following data will be recorded, noting that some data may not be available on certain hardware platforms:
Current hour: 14
Current minute: 26
Table black_box
Source Black box
CPU usage(decimal:%) 34.3
MEM usage(decimal:%) 22.1
SWAP usage(decimal:%) 0.1
DISK usage(decimal:%) 24.3
Ardexa Broker DNS ping(keyword) good
8_8_8_8 IP ping(keyword) no attempt
Gateway ping(keyword) no attempt
NS Lookup broker_ardexa_com using 8_8_8_8(keyword) no attempt
Ardexa running(keyword) True
Agent run state(keyword) True
Agent cloud connection(keyword) True
Agent message count(integer) 126435
Agent loop count(integer) 1
Agent core loop state(keyword) True
Agent comm loop state(keyword) True
Agent core ticker(integer) 667703
Agent comm ticker(integer) 801399
Agent message cache(integer) 0
Timezone(keyword) Australia/Sydney
CPU temp(decimal:°C) 55.8
Uptime(decimal:h) 2906.62
Ardexa PID(integer) 3803
Delete data(keyword) False
In addition, the following data is dumped to a file for later forensics. This "dump" file is located at /var/log/ardexa-black-box-dumps.log, and will record data as follows:
  1. 1.
    if CPU exceeds 60%, it will dump the top 4 processes by CPU usage
  2. 2.
    if MEM exceeds 60% or SWAP by 15%, it will dump the top 4 processes by MEM usage
  3. 3.
    all connected USB devices are dumped every 5 minutes
  4. 4.
    once a day, the contents of /etc/resolv.conf is dumped
The dump files are archived on a daily basis and kept for about 3 months. Dump files older than 3 months are deleted. If the black box detects that more than 75% of the disk has been used, it will attempt to delete data from the logging area (/opt/ardexa/logs). If this area uses less than 5% of the disk, data will not be deleted. Only the oldest month/year will be deleted in a 5 minute run.

Troubleshooting

Follows these steps to troubleshoot this plugin
  • Check that version 2.0.0 or greater is installed, as follows:
pip3 list | grep ardexa
  • The command ardexa_black_box will run the black box and log data to the latest file. Check it is being received by running the command: cat /opt/ardexa/logs/black_box/Black\ box/latest.csv
  • Ensure an entry in the /etc/cron.d directory, under a filename called ardexa-black-box. Cron will be used to run the 5 minute runs. The Ardexa agent is NOT used for the scheduled run.
Copy link
Outline
Purpose
Usage
Troubleshooting