We understand the importance of not just reacting to threats but proactively seeking them out. Recently, we gained access to systems suffering from persistent server reinfections, providing invaluable insights for our Threat Management team. In this blog post, we'll delve into the results of our investigation, shedding light on how we uncovered and halted these persistent reinfections.
Our journey into preventing reinfections began with a careful examination of server activities. We noticed that cron jobs caused many reinfections. To address this, we combed through the `/var/spool/cron` directories but found no malicious cron settings for the first time. However, it’s important to note that this situation changed over time, as we will explain in more detail later.
Next, we turned our attention to zombie processes. We executed the `ps aux | grep php` command, sifting through the output to identify long-running PHP processes. Normally, PHP processes should terminate quickly, so those running for hours or even days raised red flags. We particularly focused on processes referring to non-existent PHP files, such as 'style2.php,' 'plugin-install.php,' 'fqgt.php,' 'zany.php,' and 'httpd.conf.' By terminating these processes, we were able to halt reinfections on eight out of twenty servers. This common issue proved to be both identifiable and easily stoppable.
Some malicious processes proved harder to catch. These phoenix processes appeared briefly and then vanished, making them challenging to trace. They created copies of themselves, executed them, and terminated the parent process. When confronted with processes referencing ever-changing file names that do not exist upon inspection, it's a clear sign of this specific threat. To effectively address it, proactive steps have to be taken to temporarily revoke write permissions from the affected directory. This action not only hinders the execution of these processes but also enables a closer examination of their underlying code. This approach proves especially valuable when dealing with previously unidentified threats, as it allows for the generation of distinctive signatures for future identification and prevention.
In a variation of the previous threat, we encountered processes that created random-named folders and 'index.php' files within a selected directory. These processes mirrored the behavior of their counterparts, deleting files and folders, creating new ones, and executing them. By removing execution permissions from the main directory, we were able to halt this type of reinfection. Even after restoring execution permissions, the processes did not restart.
Lastly, we revisited cron jobs. If a cron job merely writes a file without launching a process, it can run every minute, causing 60 reinfections per hour. We encountered this issue on a few servers. For instance, after gaining access to one server, we cleaned up many problematic cron jobs, resulting in the cessation of 40 reinfections in just about 5 minutes.
By gaining access to servers affected by reinfections, our Threat Management team successfully stopped and investigated these persistent threats. Our journey has not only led to successes but also provided valuable insights. We've learned that reinfections can be caused not only by malicious activities but also by software conflicts, such as clashes with antivirus programs, caching systems, statistics tools, and loggers. These experiences have enriched our understanding and provided valuable lessons for comprehensive reinfection prevention.
Through adaptability and a proactive approach to cybersecurity, we remain one step ahead of the ever-evolving cyber threat landscape.
| What are server reinfections, and why do they pose a significant challenge in cybersecurity? Server reinfections occur when a cleaned server is infected again, often by the same malware. This cycle shows that initial defenses were insufficient, and the malware can effectively hide or exploit ongoing vulnerabilities. This constant cycle is challenging because it undermines server security, risks data, and requires resource-intensive efforts for re-cleaning. How do cron jobs contribute to the problem of server reinfections? Cron jobs can be used by malware to execute recurring tasks, such as downloading and running malicious scripts. These automated tasks can help the malware persist and reactivate post-cleaning, making detection and complete eradication difficult. Besides malicious activities, what other factors can contribute to server reinfections? Other factors include software conflicts (like those with antivirus programs), outdated software, weak security measures, human errors in configuration, and unresolved vulnerabilities. These factors can leave servers vulnerable to repeated attacks and reinfections. |
