In the annals of “Everyday is a learning day”. I have an Apache server hosting student websites, and the number of websites grew to over 700 within a day and Apache was complaining with error:
unable to open logs
It took a little while to narrow down the problem as I assumed there was a path that did not exist or a type but eventually noticed that it was only when the number of virtual hosts increased over 504 (each with an error log and access log so approx 1008). This is very close to 1024 and triggered the thought that the problem was related to the number of open files and ulimit.
I could check the maximum number of open files with:
ulimit -S -n
And I increased the number of open files temporarily with:
ulimit -S -n 2048
When that had solved the problem I permanently changed the setting by creating a file:
/etc/security/limits.d/88-nofile.conf
With contents:
* soft nofile 4096 * hard nofile 4096
