I recently dealt with a server that stopped adding to its logs. It was as if the logs were frozen. After a lot of investigating it was found that the server had run out of memory.

Troubleshooting

Check your server space with the following command:

df -h

Investigate where the main memory is being used. This is very often in the /var/log/ folder.

If you would like to see the sizes of the files in a folder use this command:

ls -l  [folder]

eg: ls -l /var/log

If you would like to the see the sizes of folders:

du [folder]

eg: du /var/log

Clear up space by deleting files and folders

To delete a file use the following command:

sudo rm [filename]

To recreate a blank file use the following command:

sudo touch [filename]

Unable to find what is using server space

If you try to delete a file that is still in use by a service, the file will not delete off the server, but also not be displayed in the folder structure.

To investigate if there are any deleted files still in memory use this command:

lsof | grep ‘(deleted)’

You will see a list of files still in use with an id.

Use the following command to “kill” the file using the indicated id value:

kill -9 [id]