A few days ago I spotted this forum post on Linode Forum — someone’s Nginx to PHP FastCGI set up kept on throwing 502 Gateway Error after a few days.
I have a mystery and I’m not sure how to solve it. My 360 Linode is running several WordPress-based sites using nginx. Everything is great, except I find that, every few days, fastcgi fails and I start getting 502 gateway errors when I try to post a comment or upload an image, etc. Restarting fastcgi fixes the problem.
It looks like a very similar issue I had with Ubuntu 8.04 (and PHP 5.2.4), which was fixed when I migrated to Debian 5 Lenny. It appears that the OP is indeed running the problematic Ubuntu 8.04. Well. I still have no idea what the real issue was (thus do not know how to report a bug), except my PHP FastCGI process will crash multiple times during busy time of the day (but it’s only 2-3 requests/second average).
Before switching to Lenny, I have tested multiple methods to keep my PHP FastCGI processes alive, including running check/restarting script every 5 minutes, etc. However at the end even 5 minutes downtime felt a bit excessive, so I wrote a simple daemon in Python to keep an eye on Nginx’s error log, and restart PHP/FastCGI processes whenever something bad has been detected. The problem here is that the PHP/FastCGI processes actually do not die — Nginx simply cannot communicate with them. So I need to restart PHP whenever Nginx splits a dummy.
I put the code on Linode Pastebin here. Linode has appeared to remove that script from their Pastebin, but you can Download it here, modify some variables in the code (for example the email addresses, start/stop script location, etc), and then run it as root.
What it does:
- It will damonize first.
- Scanning
/var/log/nginx/error.logevery 2 seconds, using similar algorithm astail -f. - If “104: Connection reset by peer” or “111: Connection refused” is detected in the log,
- Run
killall -9 php-cgi - Run
/etc/init.d/spawn-fcgi-php startto restart the PHP FastCGI processes
- Run
It should use minimum memory + CPU/IO (around 3.5MB RSS, and stat calls are pretty fast). At least it keeps me peace at night that I know if my PHP processes are crashing, there’s a guardian to bring them back up. It still runs on my VPS at Linode — even though with Debian 5 I am getting crashes once every 2 weeks instead of 4-5 times a day.
Hopefully it will be useful to some.
