Bypass The Anti-Virus - Part 1: Introduction

8 minute read


AV (Anti-Virus) most of the common and widely used endpoint security softwares exist that almost used on both of individuals & corporation Endpoints.Therefor, It’s a challenge for penetration testers/ethical hackers to run their tools/exploits “Which Consider as malicious softwares”. As a results, The AV software would be detecting the tools/exploits. But, Attackers are getting more smarter day by day. So, in this series we will discover and know about AV (Anti-Virus) software & how to bypass it from scratch moving to more advanced ways.

Malwares & Anti-Virus History

Viruses are the common keyword used between people. But, actually viruses are one type of the malware families & types. So, what is “malware” ?. It’s a short term for “Malicious Software” as it’s name describing it as a coded and designed softwares to perform harmful activites on the system for example ( Unauthorized Access, Delete Important files, Lock the system, Encrypt Your data and many more). You might think malwares started from 10 or 20 years ago or as something new. But, The first malware came to life was on the 70’s specifically 1971 that year when The Creaper created and it was the first malware. And it was not performing any malicious or harmful activities. In fact, It was an experiment. So, you may be wondering what was The Creaper doing ?. Basically, It was just displaying a message to challenge the use to catch it. At that time also The Reaper was created and it was the first Anti-Virus “Anti-Malware” and the goal of creating it, Was to hunt for The Creaper malware and stop it. Since that time until now there were a lot of updates on the malwares from the attackers and on the endpoint security solutions from the vendors.

How Anti-Virus Works ?


Anti-Virus is a software like any other software we use. Therefor, it requires a lot of files and libraries to be loaded so the anti-virus software can work probebly and each got different functions and used in a spasific jobs. For example the file could be for Configuration,Database,Logging&more. We gonna know more about these files & libraries and how we could exploit them in this series. But the most important part here is to know how the anti-virus can detect and catch the malware and basically, the Anti-Virus dealing and scanning with the malware through the engines and we gonna know about each one of these engines & how each one of them scan and deal the malware in different ways.

Anti-Virus Engines


Now, Time to discuss the engines of the Anti-Virus and see how each of them work:

Static Engine

First is the Static Engine and as we can guess from the name it’s scanning the malware statically by comparing it to a database of signatures that related to previous known malwares and check if there is a match with any of the signatures. Let’s take a deep look on the process.

Untitled Diagram.drawio.png

Explaining the process in the above picture. We have the file we want to scan, The anti-virus take it and pass it or gonna use the Static Engine aganist it. Then, the Static Engine start to get the signatures and collect it all from the Signatures Database/File. After that, It will start to searching with the Signatures & comparing it to the file. And if any matched signature found, It will mark the file as Malicious. Otherwise, It will mark the file as Benign which means non-malicious. So, You may got one of the ways to bypass this engine. By changing the malware form or code or anything could be exist in these signatures to avoid the detection from this engine. We will be talking about that in the coming parts.

Heuristic Engine

The Heuristic Engine is closer to the Static Engine but instead of using signatures. It’s using defined rules and these rules more close to the behaviour of the software and how is it acting and what it is doing. For example some Rules are If the software open outbound connection or Try to access some files in the system32 directory. If one of these rules matched with the software behaviour then it will be marked as a malicious software.

Untitled Diagram.drawio1.png

So, the process is as the following: The anti-virus take the file and pass it to the Heuristic Engine . Then, the Heuristic Engine start to collect the pre-defined rules . After that, It will start look into the file behaviour with these rules. And if any matched rule found, It will mark the file as Malicious. Otherwise, It will mark the file as Benign .

Unpacking Engine

This engine is not actually to scanning the file to see if it’s Malicious or Benign. So, what does it do ?. to understand the Unpacking Engine we have to know what is Packing & Unpacking Actually.

  • What is Packing & Unpacking ? Packing or Software Packing is a method that used to compress the actual code for the software and this method used to protect the code from being reversed for the normal softwares . So, the Threat Actors Used the same way to protect their malwares and make it hard to be analyzed. If we executed the Packed file it will be loaded in the memory and during the RunTime process the file get Unpacked and executed. Read More Here for deep understanding.

Let’s move on to the Unpacking Engine process. But, before we go for it, You have to know that each Packing process is done by a Packing Softwares and each software Pack the file with different algorithm And to Unpacking the Packed file you without having the Packer it will be so hard. So, the malware analyst that analysis the malware start to Unpack it manually and then write a script to Unpack it. After that add it to the Unpacking Engine .

Untitled Diagram.drawio 2.png

So, here we got the Packed file the anti-virus will pass it to the Unpacking Engine. Then, the Unpacking Engine gonna to check the Packed file to get the Packing Software that used to perform the packing. And finally it will perform Unpacking if it’s exist in the engine and give us the Unpacked File. Finally it’s the time to scan the Unpacked File.

Dynamic Engine

Finally, the Dynamic Engine. As the engine name says, It’s analyzing the Malware or the targeted software dynamically By Running the targeted file inside a sandbox and start to analysis it’ behaviour and see if it’s malicious or no. So, let’s dig more deep into the dynamic engine. The Dynamic Engine has more than 1 way to scan for the malware as the following:

  • Sandboxing So first what is Sandboxing ? It’s a process of running the file/software inside a virtual testing environment. Therefor, the code or the software would be run will be separated from the real environment “Operating System”. In this way if the software is malicious it won’t affect your “Operating System”. As a result the Dynamic Engine will be able to know if its malicious or no through its suspicious behaviour.
  • API Monitoring What is windows APIs ? It’s a set of functions that used by softwares/programes to perform some tasks such as Crreating New Files,Delete Files,ReadFiles & many more. There are some of these APIs commonly used by threat actors malwares. So, the Dynamic Engine is inspecting the API calls of the software and analysis it, looking for some common used API calls that widely used by the malwares. You can see the common malwarews API calls from Here.

Untitled Diagram.drawio 4.png

As the process shown, The anti-virus take the file and pass it to the Dynamic Engine. After thatDynamic Engine will start to analysis the file by Sandboxing/ API Monitoring after that It will mark the file as Malicious Based on the analysis. Otherwise, It will mark the file as Benign which means non-malicious.


Overall, Nowadays we can see a massive updates in security solutions such as implementing the Artificial intelligence & Machine Learning. Which making it harder for attackers to bypass the solution or the AV software. Finally, as the solutions getting smarter also, the attackers getting smarter too with their Techniques & Attacks. In the coming parts we will discuss about the Common Misconfiguration, Techniques & Attacks to bypass the Anti-Virus.