# Ardexa Control Plugin ### Overview 1. The `control-ardexa` plugin is able to undertake real time control of Modbus and OPCUA enabled devices. In broad terms, the control plugin undertakes these features: * It will set one or more outputs based on one or more prioritised inputs. No feedback control will be undertaken. * The inputs are checked by a main program thread. The inputs can either be a file, OPCUA or Modbus based inputs. * Each output is controlled by each own thread. Outputs can be `Sunny TriPower 110-60`, `Sunspec`, `Solarmax`, `ABB PVS 800`, `ABB Trio Modbus`, `Aurora`, `ABB Pro 33`, `Huawei Smart Logger`, `Huawei Sun 2000 KTL`, `Sungrow` or `Bluelog Logger`. * The loop interval, at which the input(s) are read is set by a control file. The control file is formatted as a YAML file. * The outputs will be read at the loop interval, or at multiples of the loop interval (eg; every third loop interval) * All activity is logged to the `control` table in the cloud, and errors are sent to the `control-errors` table. 2. The plugin will only run as a service. Only one instance of the `control_ardexa` binary can run at any one time. This is controlled by a lock file. 3. Ideally, the control plugin can be run as any Linux user. Ardexa will run the control plugin as root. ### Activating The control plugin is installed via the normal plugin installation process. It is activated via Ardexa Front End. In doing so, remember the following: * Only a single instance of the plugin should be implemented. Do not enable any more than one. * Control of the plugin is enabled by the control file. The Ardexa Front End will not be used (for now) to configure the control file. * Once a new control file is uploaded, the plugin will detect this new file and immediately restart itself. There is no need to issue a service restart. The file must be called `control.yaml`. Use the upload entry in the Ardexa Front end to send the new file. * The frequency in the `scenario` at the Ardexa Front End is ignored. You can enter any value. * If, for any reason, they control file cannot be read, the control plugin will go into a simple loop state; where it will not monitor any inputs or outputs. The audit trail will log an entry as `NoConfig` every 60 seconds, if a valid YAML file is not found. ### Implementation Checklist Undertake the following activities when implementing the control plugin: 1. If any of the endpoints are on a serial line, then all of the inverters on a serial MUST be controlled by a single "output". See the config file below for examples. 2. Ensure that ALL monitoring plugins that use the same serial lines in the YAML, have the serial lock argument turned on. This is must for serial lines. It is not required on TCP lines. 3. Install the control plugin, but do not create a scenario, for now. 4. Load the YAML file to the `/tmp` directory, then make sure the it checked out ok. Run the command: `control_ardexa -y -c /tmp/control.yaml`. Fix any errors. 5. Create a single scenario for the control plugin, and then upload the yaml file. 6. Wait a minute or so, then run DISCOVER to check its working ok. 7. It is **strongly recommended** that a full control test on ALL inverters be undertaken, as part of the commissioning testing. There are a number of inverters that will prevent or delay remote control unless a front panel change is undertaken. The commissioning test will fully evaluate the control capability. ### Control File 1. An example control file is shown below ``` # comments are ignored # These next values are "Global" values; they apply to all inputs and outputs. They are all optional, with defaults discussed below loop_interval: 800 output_read_check_multiple: 2 attempts: 2 debug: 0 mode: on # Each Scenario, Input AND Output must have a unique number. Numbers for inputs and outputs are unique within a scenario # There MUST be at least 1 input and 1 output for each scenario Scenario: 1: inputs: 1: # A FILE input must have a "endpoint" and a "priority" defined endpoint: /opt/ardexa/config/file_input priority: 1 type: FILE 2: # A MOXA input must have an "endpoint", "priority", "type", and "setpoint"values" defined # The "setpoint"values" are defined as "Modbus value: Percent Setting", for Register 48. endpoint: 127.0.0.1 port: 1401 priority: 2 type: MoxaE1214 setpoint_values: 0 : 100 1 : 0 2 : 30 4 : 60 outputs: 3: # An output must have an "endpoint", "priority", "type", and "control" defined. All else is optional. endpoint: 192.168.7.11 port: 502 type: SMASTP110 control: percent_active_power addresses: "3" timeout: 455 4: # Aurora is allowed to use a serial line. In such cases, a "port" is not required. # Unless otherwise advised, these arguments are available for serial ports. All are optional with defaults shown: # baudrate: 9600, # databits: 7, # stopbits: 0, # parity: odd endpoint: /dev/ttyS0 type: aurora addresses: "3,4,7-9" control: percent_active_power timeout: 1780 2: inputs: 1: endpoint: /opt/ardexa/config/file_input2.yaml priority: 1 type: FILE 2: endpoint: 128.0.0.1 port: 1401 priority: 2 type: MoxaE1214 # Defined as "Modbus value: Percent Setting" setpoint_values: 0 : 100 1 : 0 2 : 30 4 : 60 outputs: 3: endpoint: 128.0.0.2 port: 502 type: BlueLog control: percent_active_power addresses: "4" timeout: 1000 4: # The "post_connection_delay" token only applies to the "sun2000ktl" (Huawei). For all other ouput types, "post_connection_delay" will be ignored endpoint: 128.0.0.3 port: 502 type: sun2000ktl control: percent_active_power addresses: "10" post_connection_delay: 1000 5: endpoint: /dev/ttyS1 type: abbpro33 control: percent_active_power addresses: "10-23" timeout: 500 baudrate: 19200, databits: 8, stopbits: 1, parity: none 6: endpoint: 192.168.3.4 type: sunspec control: percent_active_power addresses: "13,14,15,1,3,5" timeout: 500 7: endpoint: 192.168.3.45 type: sungrow control: percent_active_power addresses: "13-19" timeout: 500 ``` ### A note about Serial Connections 1. Serial connections will need to have controlled access to the serial line. This means that of there is an Ardexa monitoring plugin running on the same serial line, it will need to be controlled via PIDFILE. This is offered as a "serial lock" argument in the Ardexa plugins. The "serial lock" argument **MUST BE USED WHEN USING THE CONTROL PLUGIN OVER A SERIAL LINE**. If not, then the control plugin and the monitoring plugin will start "talking over the top of each other", which will result in unpredictable effects. 2. All serial reads and writes will delay for 100 milliseconds before undertaking a read or write. ### Global Settings 1. The `Global Settings` applies to all inputs and outputs of the control plugin. These global settings are defined in the control file at the same level as the `scenarios` key, are as follows. If they are not defined in the control file, the default values specified below will be used. * `loop_interval`: This is a value in milliseconds. It defines the rate at which all the inputs are read. The loop interval maximum (slowest) is 120,000 milliseconds (2 minutes). The fastest can be 0 milliseconds - although this is not recommended. If not defined, the loop interval is defaulted to `1000` milliseconds. **Make sure the loop interval is set to be a value higher than the normal response rate of the slowest output**. Also; **Do not set a loop interval that is significantly quicker than what is required. This will simply waste resources**. Each input and output operates from a "thread" - which means they have their own independent program space for execution. If a thread heartbeat is not received within a predetermined time, the thread will be deemed to have failed and will be restarted. This will be logged to the `control-errors` table. See below. * `output_read_check_multiple`: This is a value that defines how often to read the **outputs**. An output\_read\_check\_multiple of `10` means read the outputs every 10th loop interval. Whereas a value of (say) 3 means read the outputs once every 3 loop intervals. When there is a change to the input, the outputs will be changed as soon as possible but no later than the loop interval, that is; when an input is changed and an output needs to be changed to meet this new input, the `output_read_check_multiple` is ignored. The output\_read\_check\_multiple maximum is 100. If not defined, the output\_read\_check\_multiple is defaulted to `10`. **Do not set an "output\_read\_check\_multiple" that is significantly quicker than what is required. This will simply waste resources and will interfere with normal collection** Outputs do not need to be read at very frequent intervals. The plugin will check and reset an inverter if it does not meet the required setpoint. * `attempts`: This is a value that defines how many attempts to try and read the inputs and outputs. `Attempts` will be ignored for `File` inputs. The attempts maximum is 100. If not defined, the attempts is defaulted to `2`. * `debug`: This is a value of `0` (no Debugging), `1` (some debugging) or `2` (extensive debugging). It defines the debugging level. Debug entries are not sent to the cloud due to the amount of logs they can generate. Errors and alerts are sent to the cloud (read below). If not defined, the debug is defaulted to `0`. Only change the debug level when directed by the Ardexa Development team. **Make sure the debug entry is turned off (to `0`) when debugging is no longer required** * `mode`: This can be defined as `on`, `off` or `noconfig`. In normal operation, this should be defined to `on`. In `off` or `noconfig`, the plugin will not read any inputs or outputs. It will continue to poll at the loop interval, and log audit entries. If not defined, the mode is defaulted to `on`. Remember the following when setting a `loop_interval` and `output_read_check_multiple`. The `loop_interval` can be thought of as the rate at which inputs are checked. `output_read_check_multiple` is for outputs, and can be the same as the inputs or slower. If an input is changed, the outputs will be notified **as soon as this change is detected**. Furthermore; the control plugin will check at regular intervals that the setpoint is set as required. If it is not, it will reset them. There is no need to regularly check the outputs at a very high frequency rate. This will only use resources unnecessarily. 2. `Scenarios`. The YAML file example shows two scenarios. Any number of scenarios can be defined in the YAML file. They must have their own unique number. Each input and output within a scenario must also have its own unique number. These numbers are used for logging and debugging, and identify the threads. Inputs or outputs cannot be shared with other scenarios. That is to say, any endpoint/port pair in any single scenario, cannot appear again in that scenario, NOR any another scenario. ### Input Settings 1. `Inputs`. Each input must have a unique endpoint, port and priority number. The lowest number means the highest priority, and so the highest priority is "1". Allowed input `type` include: ``` a. MOXA E1214. To use this input include the string "moxae1214" in the config file. b. FILE. This is a standard file input. A YAML file is not required . To use this input include the string "file" in the config file. When setting percentage or absolute values, only integers are to be used. Floating point numbers will be ignored. ``` Each MOXA must have a `setpoint_values` defined that maps the register 48 value to a percentage figure. The map must use integers only. Each input can have a timeout defined, in milliseconds. This timeout is the maximum time the plugin will wait for the input to respond. If it responds earlier than this time, then the plugin will not wait for the maximum time, it will return as soon as the data is read. The default timeout value if not defined is `1000`. ### Output Settings 1. `Outputs`: Each output must have an `endpoint`, `priority`, `type`, and `control` defined. All else is optional. Allowed output `type` include: * `Bluelog Logger`. To use this output include the string `bluelog` in the configuration file. The Bluelog logger only allows setting of active power as a percentage value so be sure to only use the `percent_active_power` for the `control` type. The Ardexa Control plugin will set the watchdog (timer) and the validity time on the logger. Bluelog logger can only be contacted via TCP, so serial `endpoint` will not be accepted. For this output, you MUST Have an `addresses` entry of `"10"` * `Sunspec` enabled solar inverters. To use this output include the string `sunspec` in the `type` argument for the output. Sunspec inverters only allows setting of active power as a percentage value so be sure to only use the `percent_active_power` for the `control` type. Connections via TCP or serial will be accepted. The Ardexa control plugin will set the `Connection Control` and `Active Power Control Enable` registers, if they are not already set. For Sunspec control, the `WMaxLimPct_RvrtTms` register will be set to `0` by the control plugin. This means power changes will not timeout. For `Fimer` PVS 100 and related inverters, the front panel must be changed to allow Sunspec control. On the Fimer, this setting is `Settings -> Active Power Control -> Active Power Curtailment En/Dis`..and set to ENABLE. Documentation says: ``` By accessing to the Admin Plus sub-menu you can obtain the “Admin Plus” user privileges. This allow you to: - Change the grid standard of the inverter, after 24 hours while the inverter is operating ( so the Country Standard sub-menu on TOOLS section is locked). - View and edit the AC Settings, Reactive Power Control, Ramp Control, Active Power Control and Frequency Control: P(f) - View and edit the “MPPT Noise amplitude” field in DC Settings on SETUP section. To access on the Web User Interface with the “Admin Plus” user privileges it’s required to enter a security token that can be obtained by registering on the website https://registration.abbsolarinverters.com. Refer to the dedicated section on this topic in the manual. ``` Also note that for Kaco inverters using the Sunspec protocol, there are inverter panel settings that may delay or prevent the control signals from changing the output. These are stated as `Output gradient limitation increase` or `Output gradient limitation decrease` or similar. These settings cannot be changed by the control plugin, and are not able to be controlled by Sunspec. They must be changed on the front panel of the inverter. Please note the Implementation Checklist (above) that states: ``` 7. It is **strongly recommended** that a full control test on ALL inverters be undertaken, as part of the commissioning testing. There are a number of inverters that will prevent or delay remote control unless a front panel change is undertaken. The commissioning test will fully evaluate the control capability. ``` * `Sunny TriPower STP 110-60`. To use this output type include the string `SMASTP110` in the `type` argument for the output. Sunny TriPower STP 110-60 inverters only allows setting of active power as a percentage value. Connections via TCP or serial are accepted. **IMPORTANT NOTE**: The Ardexa Control plugin **WILL NOT** write values to EEPROM (only RAM memory), and in any case will only write values to the inverter when it is absolutely required to meet a setpoint command. From the SMA documentation: `...Except of model 123 (Immediate Inverter Controls) all data is written to flash memory - i.e. excessive write access can damage the device. As the data in model 123 is stored in RAM, it does not get restored after inverter shutdown. Cyclical writing is required....`. Ardexa WILL NOT set the default (failover) register. It will only write to the dynamic register. **THIS IS IMPORTANT**: Due to the EEPROM limitations, you must set registers 40233 and 40238 to a value of `1`. Use mbpoll as follows to check the values: `mbpoll -m tcp -a 1 -r 40233 -c 6 -t 4 -1 -B -p 502 192.168.0.11`. If they need to be set, use the mbpoll commands as follows, but only set them if THEY ARE NOT already at a value of 1 ``` mbpoll -m tcp -a 1 -r 40233 -t 4 -1 -B -p 502 192.168.0.11 1 .... for Register 40233 mbpoll -m tcp -a 1 -r 40238 -t 4 -1 -B -p 502 192.168.0.11 1 .... for Register 40238 ``` * `ABB Aurora`. To use this output type include the string `aurora` in the `type` argument for the output. Aurora inverters only allows setting of active power as a percentage value. Connections via TCP or serial are accepted. Serial connectivity is hard coded to 19200 baud, 8 data bits, 1 stop bit and no parity. Therefore, there is no requirement to include the serial tokens in this output type. Aurora uses address and so an `addresses` entry should be included, otherwise a default of `1` will be used. The plugin will ask the inverter to respond to a change in setpoint within 1.2 seconds of the command being issued. The Ardexa Control plugin **WILL NOT** write values to EEPROM (only RAM memory) and in any case will only write values to the inverter when it is absolutely required to meet a setpoint command. From the ABB documentation: `...Due to the fact the allowed number of writings on EEPROM is limited, this method shall not be used in case of dynamic control methods....`. Ardexa WILL NOT set the default (failover) register. It will only write to the dynamic register. * `Huawei Sun 2000 KTL`. To use this output type include the string `sun2000ktl` in the `type` argument for the output. Huawei Sun 2000 inverters allow setting of active power as a percentage or an absolute value. So the `control` type can be either `percent_active_power` or `absolute_active_power`. Only connections via TCP are accepted. Huawei uses addresses, and so an `address` entry should be included, otherwise a default of `1` will be used. A `post_connection_delay` argument is a MUST since the Huawei SDongles have a serious software flaw that means a delay of 2 seconds must be enacted between connecting to a server, and reading data. The `post_connection_delay` values of at least 3000 (milliseconds) should be used. The Ardexa Control plugin will check that values to the inverters are written permanently. This is a `0` value in register `42017`, which appears to be a default value for Huawei inverters. There is no mention of any EEPROM issues in Huawei inverters. * `Huawei Smart Logger`. To use this output type include the string `huaweilogger` in the `type` argument for the output. Huawei Smart Logger allow setting of active power as a percentage for all connected inverters. That is to say; this will control all connected inverters via a single Smart Logger. The `control` type must be `percent_active_power`. Only connections via TCP are accepted. Huawei Smart Logger uses an address of `0`. Any other address will result in an error. The Ardexa Control plugin will check that values to the logger are written permanently. There is no mention of any EEPROM issues in Huawei inverters. * `ABB Pro 33`. To use this output type include the string `abbpro33` in the `control` argument for the output. ABB Pro 33 inverters allow setting of active power only as a percentage value. So the `control` type can only be `percent_active_power`. Connections via TCP or Serial are accepted. ABB Pro 33 uses address and so an `address` entry should be included, otherwise a default of `1` will be used. There is no mention of any EEPROM issues in ABB Pro 33 inverters. For Serial connections most installations seem to use Recommend Use 19200 baud rate, 8 data bits, No parity and 1 stop bit. But check the installation. * `ABB PVS 800`. To use this output type include the string `abbpvs800` in the `control` argument for the output. ABB PVS 800 inverters allow setting of active power only as a percentage value. So the `control` type can only be `percent_active_power`. Connections via TCP or Serial are accepted. ABB PVS 800 uses address and so an `address` entry should be included, otherwise a default of `1` will be used. There is no mention of any EEPROM issues in ABB PVS 800 inverters. Note that: a. Register 3116 is used for control. This will be scaled by 100. So a 100% power setting = 10000 b. Register 3110 will not be touched by the control plugin * `Sungrow`. To use this output type include the string `sungrow` in the `control` argument for the output. The control plugin only allows control setting of Sungrow by active power only as a percentage value. So the `control` type can only be `percent_active_power`. Connections via TCP or Serial are accepted. Sungrow uses address and so an `address` entry should be included, otherwise a default of `1` will be used. There is no mention of any EEPROM issues in Sungrow inverters. Note that: a. Register 5008 is used for control. This will be scaled by 10. So a 100% power setting = 1000 b. Register 5007 will be set to "Enable (170)" by the control plugin, if its not already set * `Solarmax`. To use this output type include the string `solarmax` in the `control` argument for the output. Solarmax inverters allow setting of active power only as a percentage value. So the `control` type can only be `percent_active_power`. Connections via TCP or Serial are accepted. Solarmax uses address and so an `address` entry should be included, otherwise a default of `1` will be used. There is no mention of any EEPROM issues in Solarmax inverters. For Serial connections most installations seem to use Recommend Use 19200 baud rate, 8 data bits, No parity and 1 stop bit. But check the installation. Also, please note the following warning from the Solarmax manufacturer. The Control plugin **WILL NOT** limit the power output of the inverters. This is up to the user of the Control plugin to manage. ``` Notice: With the command DPR the inverter will not be disconnected from the grid, even if the active power is set to 0%. The range from 0% to <8% must be used only for temporary control commands. It’s not allowed to operate the inverter in this range for more than two minutes without leaving this range again. Notice: To let the Inverter run on 0% active power reduction for a long time could lead to malfunction of the Inverter... ``` The Control plugin **WILL NOT** limit the power output of the inverters. This is up to the user of the Control plugin to manage. * `ABB Trio Modbus`. To use this output type include the string `abbtrio` in the `type` argument for the output. ABB Trio Modbus inverters only allows setting of active power as a percentage value. Connections via TCP or serial are accepted. Ardexa WILL NOT set the remote control register (Register 180). This register exists only on a small number of the Modbus Trio inverters, and the default should be `0` (remote control enabled)). Use mbpoll as follows to check the value: `mbpoll -m tcp -a {ADDRESS} -r 180 -c 1 -t 4 -1 {IP/SERIAL}`. If the value exists and it is NOT `0`, then set it to `0` as follows: ``` mbpoll -m tcp -a {ADDRESS} -r 180 -c 1 -t 4 -1 {IP/SERIAL} 0 ``` 2. Each output must or may have one of the following tokens: * `type` (MANDATORY): Must be one of `bluelog`, `Sunspec`, `smastp110`, `aurora`, `sun2000ktl`, `solarmax`, `abbpvs800`, `abbtrio` or `abbpro33` * `endpoint` (MANDATORY): Must be included as an IP address or serial device. * `port` (MANDATORY): A port is mandatory ONLY for an IP address (not a serial device). A port entry must be a number between 0 and 65535. * `control` (MANDATORY): This will be either `percent_active_power` or `absolute_active_power`. But either of these can only be used on some inverters. Check the documentation above. * `timeout` (OPTIONAL): Each output can have a timeout defined, in milliseconds. This timeout is the maximum time the plugin will wait for the output to respond. If it responds earlier than this time, then the plugin will not wait for the maximum time, it will return as soon as the data is read. The default timeout value if not defined is `1000`. * `addresses` (OPTIONAL): For outputs that use Modbus (including Sunspec), an `address` value may be defined. The default is `1`, if an `address` entry is not included with the output. It can be less than `1` nor greater than `255`. An `address` included for outputs that do not use them will be ignored. If serial lines or TCP endpoint hosts many addresses, then include the range of addresses. Examples include "2-4", or "1,2,3,5-9". The addresses entry **MUST BE QUOTED** * `post_connection_delay` (OPTIONAL): This is only used for Huawei Inverters. The default value is to have no delay. If using the Huawei SDongle, make sure this setting is set to at least `3000` or more. * `baudrate` (OPTIONAL): This may be set to a value of say `9200`. It is only used for serial connections and will be ignored for IP Addresses. The default is `19200` * `databits` (OPTIONAL): This may be set to a value of say `7`. It is only used for serial connections and will be ignored for IP Addresses. The default is `8` * `stopbits` (OPTIONAL): This may be set to a value of say `0`. It is only used for serial connections and will be ignored for IP Addresses. The default is `1` * `parity` (OPTIONAL): This may be set to a value of `even`, `odd` or `none`. It is only used for serial connections and will be ignored for IP Addresses. The default is `even` * `pty` (OPTIONAL): Flags the serial connection as a "pseudo terminal" (mostly used in testing). This may be set to a value of `true` or `false`. It is only used for serial connections and will be ignored for IP Addresses. The default is `false` 3. If an output setpoint has been set, and the inverter cannot subsequently be contacted for whatever reason, the control plugin will continue to attempt to access the inverter. The previous successful setpoint set on the inverter will be assumed to still be current on the inverter. ### Control File Testing and Backups 1. The control plugin will automatically save a copy of each **unique** copy of the control file. These files will have a name something like: `2AA4C5EE_2024-09-11T14:49:08+10:00_config_file.yaml`. The checksum is the leading string `2AA4C5EE`. This checksum will appear in the audit logs (see below) and uniquely identifies the control file being used. Each time a new configuration file is copied to the control plugin, it will be copied to the archive area (`/opt/ardexa/config/control-ardexa/config_file_backups/`) ONLY IF the checksum does not already exist. The configuration file backups will be stored for the life of the system on the Ardexa cloud. 2. Configuration files are checked for errors will not be copied to the backup area. Checking for errors can be undertaken by running the command: `control_ardexa -c {CONFIG FILE LOCATION"} -y` for example; `control_ardexa -y -c /tmp/location/control.yaml` ### Discovery 1. The operation of the plugin can be determined by running the `DISCOVER` button in the Ardexa front end. On the command line, this is achieved by running the command `control_plugin -d`. An example discovery is shown below: ``` Inputs: /opt/ardexa/config/control-ardexa/file_input.yaml Scenario ID: 1 Thread ID: 1.1 Input Type: File Priority: 1 Time Since Last Read: No timestamp available File Last Modification Time: 2024-10-28T10:30:37.783631542Z Current Setpoint: 10 Bad Read Count: 0 Timeout (millisecs): 1000 Contactable: true Outputs: 192.168.8.11:502 Scenario ID: 1 Thread ID: 1.2 Output Type: SMASTP110 Control Type: PercentActivePower Address: 1 Timestamp: 5 seconds ago Time Since Last Heartbeat: 1 seconds ago Current Setpoint: 10 Timeout (millisecs): 1000 Contactable: true ``` ### Logging 1. There are 4 levels of reporting enabled on the control plugin, `high_frequency` data collection, `audit` logging, `error` logging and `debug` logging. 2. `audit` logs are generated once every 60 seconds and whenever any setpoint is changed. A log entry will be made for every input and output. Each log entry will be sent to the cloud and appear in the `control` table. Audit logs contain the following fields: ``` datetime: date ... This is the datetime of the audit log scenario_id: String ... This is the scenario ID. Something like: "1" thread_id: String ... This is the thread ID. Something like: "1.3" ("1" is the parent scenario ID) mode: ModeType ... Will "On", "Off" or "NoConfig" endpoint: String ... This will be the endpoint string. Something like "127.0.0.1" port: String ... This is the TCP port (if it exists). Something like "502" address: Integer ... The address of the endpoint endpoint_type: String ... This is the type of endpoint. something like "BlueLog", "SMASTP110" or "Sunspec" control_type: String ... If the log record refers to an output it will one of "ActivePower", "ReactivePower" or "PowerFactor". Otherwise "Input" will appear action: LogAction ... This will be one of "ChangeSetpoint", "RegularReport" (that is; a 60 second report), "SetPointChanged", "SetPointNotChanged", "ReadingsUpdate" or "YAMLChanged" current_setpoint: String ... The current setpoint for the output. It may be empty if no setpoint defined. If the "endpoint_type" is "File", then the audit trail will detail the current contents of the file input, only if the file can be read and the contents translate to an integer. AC Power: Decimal ... The current AC Power (in Watts) for the output. It may be empty if no AC Power value is defined config_checksum: String ... This is the CRC32 Checksum of the config file process_id: String ... The process ID of the control plugin. Used to track the control plugin reliability contactable: bool ... "true" if the output is contactable ``` 3. `error` logs are recorded whenever an error condition (one that causes the control plugin to stop), or an alert condition (a critical condition that does NOT result in program termination) occurs. These records are sent to the cloud, and appear in the `control-errors` table. ERROR logs indicate a fatal program condition resulting in program termination. In such cases, the control plugin will be automatically restarted. Error logs contain the following fields: ``` datetime: date ... This is the datetime of the error log Error Code: String ... This will be a code like "ERROR23". This will provide the developers are key to where the condition occurred in the program Description: String ... This is a full description of the error ``` 4. `debug` logs will be recorded if the YAML file contains an entry of `debug: 1` or `debug: 2`. Debugging will generate a huge amount of logs, and will not be sent to the cloud. The logs will be written to the local machine, on the file: `/var/log/ardexa-control.log`. These logs are used by developers to fault find critical issues, and so must only be set to `1` or `2` on direction of the development staff. Debugging generates a large volume of logs, and so therefore .. **DO NOT leave debugging on more than is required**. The debug logs are compressed and rotated every day. Daily archives are stored for a maximum of 50 days. 5. `high_frequency` logs are generated once every time the output reads the AC Power and Setpoint. Each log entry will be sent to the cloud and appear in the `hi_freq` table. High frequency logs contain the following fields. The source names for these endpoints will appear in the Ardexa cloud as: a. For Serial inverters: serial device (minus the "dev") + the address. Eg; `ttyS1/27` b. For TCP inverters: IP address/Port/Address. Eg; `192_168_2_31/502/34` ``` datetime: date ... This is the datetime of the audit log endpoint: String ... This will be the endpoint string. Something like "127.0.0.1" port: String ... This is the TCP port (if it exists). Something like "502" address: Integer ... The address of the endpoint endpoint_type: String ... This is the type of endpoint. something like "BlueLog", "SMASTP110" or "Sunspec" action: LogAction ... This will be one of "ChangeSetpoint", "RegularReport" (that is; a 60 second report), "SetPointChanged", "SetPointNotChanged", "ReadingsUpdate" or "YAMLChanged" Setpoint: Decimal ... The current setpoint for the output as a percentage. It may be empty if no setpoint defined AC Power: Decimal ... The current AC Power (in Watts) for the output. It may be empty if no AC Power value is defined contactable: bool ... "true" if the output is contactable ``` ### Troubleshooting 1. The following steps must be undertaken to troubleshoot the control plugin. When reporting any issues, send all the following data to the Ardexa development team. * Check the status of the control plugin service and make sure it is running: `sudo systemctl status control-ardexa.service` * If the process is not running, check PID file: `cat /var/lock/ardexa_control.pid`. The contents of this file is the process ID. Check the process ID is running as follows: `ps -p {PID}`. It should return something like: ``` PID TTY TIME CMD 440271 ? 00:01:41 control_ardexa ``` If the process is not running, remove the PID file as follows: `rm /var/lock/ardexa_control.pid` and restart the service as follows: `sudo systemctl restart control-ardexa.service` * Check the version of the control plugin: `control_ardexa --version`. If its not the latest version, upgrade it. * Check the control file does not have any errors: `control_ardexa -y`. If you have the control file in another temporary location, use: `control_ardexa -y -c /tmp/location/control.yaml` * Check the audit log: `cat /opt/ardexa/logs/control/control-ardexa/latest.csv`, and make sure an entry is being recorded at least every 60 seconds * Check the error log for any obvious errors: `cat /opt/ardexa/logs/control-errors/control-ardexa/latest.csv` * If debug is turned on, check the debug log: `cat /var/log/ardexa-control.log`. Note: **DO NOT leave debugging on more than is required** * Check to see if the control plugin is able to discover data: `control_ardexa -d` * Check the errors on the "systemd console". Some errors can't be reported to the error or debug files. So check `sudo systemctl status control-ardexa.service` to see if there are any errors being reported in the console.