Other Agent related issue

LIMITS

Here is a one-shot REMOTE SHELL command to raise all important limits. Simply

1. Use the copy icon to copy the script

2. Paste it into REMOTE SHELL and click RUN COMMAND

3. Browse to Manual Config -> General and restart the agent

mkdir -p /etc/systemd/system/ardexa.service.d;
echo "[Service]\nLimitNOFILE=infinity" > /etc/systemd/system/ardexa.service.d/override.conf;
systemctl daemon-reload;
if ! grep -q '^fs.inotify.max_user_watches' /etc/sysctl.conf; then
  echo 'fs.inotify.max_user_watches=1048576' >> /etc/sysctl.conf;
fi;
sysctl -p;

Threadpool

For very large sites, it is recommended to increase the size of the threadpool. You can check the size of the threadpool using the following command:

bash -c 'echo $(( $(ps -Lef | grep sbin[/]ardexa | wc -l) - 12))'

To increase the size, run the following script using REMOTE SHELL and restart the agent (Manual Config -> General -> Restart Agent)

mkdir -p /etc/systemd/system/ardexa.service.d;
printf '[Service]\nEnvironment="UV_THREADPOOL_SIZE=128"\n' > /etc/systemd/system/ardexa.service.d/threadpool.conf;
systemctl daemon-reload

Disk is full

df -h

Remove some big files

Inodes are full

df -i

This means the number of files has reached its limit, not the size. Remove or tar as many files as you can to reduce the inode usage.

File handle limits

ulimit -Hn
ulimit -Sn

systemd does not honour the old "system wide" methods of increasing file limits. Instead you need to override the service specific file handle allowance. Add the following file to the directory /etc/systemd/system/ardexa.service.d

override.conf

[Service]
LimitNOFILE=infinity

Inotify limits

cat /proc/sys/fs/inotify/max_user_watches

To examine just how many file inotify watches are in use, use this script: https://github.com/fatso83/dotfiles/blob/master/utils/scripts/inotify-consumers

The Ardexa agent uses inotify to watch for changes to the file system. For very, very large sites, the default limits might be too small. Increase the limit as follows:

echo 'fs.inotify.max_user_watches=1048576' >> /etc/sysctl.conf;
sysctl -p