Dynamic Configuration

Automated plugin management and data capture

With the release of Ardexa Agent 2.3.0, Dynamic Configuration is now available for all users. This feature allows log files to be automatically parsed and the data contained within sent to the cloud in real time (if a connection is available), or to be cached so it can be sent when a connection does become available. This automatic collection of data is via a file configured as Comma Separated Values (CSV) files (see: https://en.wikipedia.org/wiki/Comma-separated_values), is called Dynamic Configuration.

For data to be collected via Dynamic Configuration, the following definition must be contained in the /etc/ardexa/ardexa.yaml file:

dynamic_tail:
  - directory: /opt/ardexa/logs
    filename: latest.csv

As of Ardexa agent 2.3.0, this defintition is included and the default file location for Dynamic Configuration is /opt/ardexa/logs. In order to collect data, a few rules must be followed.

Table and Source Definitions

Data collected to the Ardexa cloud is always sent to a particular table. In addition, a source needs to be defined that further identifies the sub-component that is sending the data. For Dynamic Configuration, at least 2 sub-directories must be defined in /opt/ardexa/logs to identify the table and source. It is allowable to have more than 1 sub-directory for the source definition. So in the following example, are all allowable:

/opt/ardexa/logs/solar/inverter1 .... table = solar; source = inverter1
/opt/ardexa/logs/vehicle/ecu1/str3 .... table = vehicle; source = ecu1/str3
/opt/ardexa/logs/hydro/gate2/pump3/value4/ .... table = hydro; source = gate2/pump3/value4

CSV File Definitions

Once the Ardexa agent encounters a file called latest.csv (as per the above definition in the YAML file), the Ardexa agent will look in these files for data. Note that you can store any number of files in any of the Dynamic Configuration directories. If they are not named latest.csv then all files will be ignored by the agent. Only one latest.csv file per directory is allowed. The latest.csv must be formatted as shown in the example below:

# AC Power (decimal:W), Datetime (date), Count (integer:), On (bool), Status (keyword)
234, 2021-02-03T12:23:32Z, 12, True, MPP track
235.6, 2021-02-03T12:23:34Z, 13, 0, MPP tracking
245.8999, 2021-02-03T12:33:32Z, 15, false, MPP track

Note the following:

• There can only be 1 header line, it must be at the top of the file and MUST start with #

• The format for each header variable is name(expect:units)

• name is mandatory. Any characters including spaces, are allowed except for dots or commas

• expect is optional. Allowable expects are as follows:

  • date defines date value. It MUST be ISO8601/RFC3339 compliant date and time plus timezone offset, e.g. 2020-12-07T14:20:00+1100

  • decimal any number with a fractional component

  • integer defines any whole number. The value cannot have a decimal/fractional component

  • bool defines a boolean value. 0 or false (case insensitive) are False, any other value is true

  • keyword defines a fixed string , usually chosen from a small pool of choices, e.g. OK, RUNNING, ERROR, etc, which can contain any characters. If commas need to be included, they must be enclosed in quotes. If expect is not defined, then this will be used as the default.

  • log: unstructured text. Good for things like "comment" fields

  • discard: this field will not be sent to the cloud

• units are optional and only apply to Numbers, otherwise ignored. When leaving out units, remove the :

If latest.csv file is not formatted according to the above, data will be rejected. If the number of values or the types does not match the header line, data will be rejected. The Ardexa cloud contains a table called agent errors . This can be perused using SEARCHES to look for errors.