I'll look at those, thanks.
In the meantime, I did start poking around a bit more to attempt to see where it was spinning. Since PHP was using such low processor time and seemingly MySQL wasn't being pounded, I thought perhaps it would be useful if we had a Mac OS X fseventer type of tool. I found a handy little doodad that works with inotify in inotify-tools called `inotifywait`.
This is exceptionally useful as you can use it to monitor any and all file activity on the entire filesystem. It certainly should become common knowledge for any admin that needs to troubleshoot things.
I did a:
`wget http://download.fedora.redhat.com/pub/epel/5/i386/inotify-tools-3.14-1.el5.i386.rpm`
`yum localinstall inotify-tools-3.14-1.el5.i386.rpm`
And then modified: /proc/sys/fs/inotify/max_user_watches to something more reasonable, like 100000
You'll want to learn more about how to use it with: `man inotifywait`
For example, I found this command quite useful and telling in my case (simply trimmed out any chatty services that weren't like to match my issue at hand):
inotifywait -rm / --exclude "/proc|/dev|dnscache|squid|samba|iptables|cron\.d|tinydns|/spool/|ntp|clamav|freshclam|MYI|MYD|sess_|access_log"
This yielded about 14,000 lines of cache activity for Concrete5 with exceedingly many entries like:
/home/e-smith/files/ibays/share_name/html/files/cache/ ACCESS zend_cache---593ee045d00a356e1adfff791e74b6551d87846f68906481b9a2a4ae035b9599
/home/e-smith/files/ibays/share_name/html/files/cache/ CLOSE_NOWRITE,CLOSE zend_cache---593ee045d00a356e1adfff791e74b6551d87846f68906481b9a2a4ae035b9599
So I believe it has something to do with some cache dysfunction for this particular scenario as it makes no sense to have this much activity for a small app.