Static Data Exploration of Malware and Goodware Samples

Two years ago, we found ourselves completely captivated by the perfect intersection of data hoarding, write-once Python, and shiny graphs as, inspired by the 2018 EMBER paper, we began gathering, parsing, and analysing malicious and benign binaries. We eventually discussed this process and our findings at Steelcon 2024 in our talk, Statistical Inconspicuousness, where we sought to identify what a new implant would have to do to blend into the contemporary landscape of goodware. Our hope was that this analysis would both inform and inspire the creation of better payloads as well as give blue teams pointers for further attributes and behaviours for static malware analysis.

Our preliminary findings were pretty cool, but out of a mix of life and burnout, we’ve been long overdue a follow-up. But last week, Ember 2024 was released as a part of EMBER2024 - A Benchmark Dataset for Holistic Evaluation of Malware Classifiers which rekindled our interest in this topic. We began updating and expanding our 2024 data set, updating our review process, and performing further analysis techniques drawing on the EMBER2024 paper, EMBER2024 - A Benchmark Dataset for Holistic Evaluation of Malware Classifiers as well as other research published since our last talk. Ultimately, what we want to do in this and in upcoming blogs is look at the state of software from a static perspective and see what we can learn to create better malware, and better ways of differentiating between malware and benign executables. We’ll look at how we collected our samples, how we created the dataset, and then explore and document our initial and heavily procrastinated-upon findings using our updated approach.

Data gathering

As with so many projects, pulling the data is the hardest part of this problem – making sure that our samples are broadly consistent and that they reflect an appropriate breadth and depth of the full population. In our processing, we differentiated our samples into two main categories before setting out to find as many examples as we could:

Collecting malware samples

We started with malware. Malware is easy to get, centralised, and available all over the Internet. While many helpful sites offer to give you malware free, one implant at a time, we found several established organisations with extensive malware collections. We collected our samples from the following datasets:

1. Sophos ReversingLabs 20 million dataset SOREL-20M
2. Abuse.ch and Spamhaus' MalwareBazaar
3. The inimitable VX-Underground

The Sophos Reversinglabs SOREL-20M dataset was and is the most convenient malware dataset which we found, as it contains (unsurprisingly) 20 million samples which were all readily downloadable from Amazon S3. However, as detailed in their blog, Sophos-ReversingLabs (SOREL) 20 Million sample malware dataset, the samples within the dataset were disarmed by stripping the Subsystem and FileHeader.Machine header values, which could potentially impact our analysis by reducing the available metadata held within the subsystem field.

For malware samples, we reviewed complete binaries, with the OptionalHeader.Subsystem flag and the FileHeader.Machine header value both set to 0 to prevent accidental execution.

Collecting goodware samples

Goodware is a lot harder to come by, validated goodware even more so. For goodware, we used a mix of sources for our talk:

1. National Software Reference Library (NSRL)
2. Various configurations, versions, and distributions of Microsoft Windows and Microsoft Windows server.
3. Typical Microsoft software, including applications, services, libraries, and other distributables.
4. Chocolatey
5. ninite

The National Software Reference Library (NSRL) database maintains a list of hashes for vendor-submitted applications. With those hashes, the challenge is to then find the file itself. For us, we found the Hybrid Analysis API worked to let us pull the original binaries. We also made use of other publicly available assistants; we some simply scripts to pull the versions of binaries from Chocolatey, the package manager for Windows, as well as (fewer) samples from Ninite, the initial installation assistant. We also grabbed as much as we could from our Windows lab environments, including several desktop and server installations of Windows, as well as many roles and services which we could install on them.

This dataset is by no means a complete representation of all available or accessible goodware samples. We should note, as EMBER do, that the licensing and distribution of goodware, unlike (most!) malware, is considerably more restrictive, and we would encourage the responsible and ethical construction of a more complete goodware dataset to be left as an exercise for the endeavouring reader.

Data Parsing

With our new samples covering both malware and goodware, we built a parser to replicate the work of ember and build a dataset with similar features. For a comprehensive walkthrough of how we approached automating the static review of the samples, see Mez0's blog, "Citadel: Binary Static Analysis Framework". In lieu of re-creating the whole blog here, our parsing aimed to review each binary and highlight features from their metadata and data, including:

1. Imports
2. Exports
3. Optional Headers
4. Sections
5. Entropy (both for the file and section by section)
6. File Size
7. Toolchains (compilers, linkers, etc)
8. Code Signing
9. TLS
10. Symbols
11. Hashing (TLSH and SHA256)

One thing we found we couldn't get was malware families. Perhaps Citadel could be updated to do this for us, but it would take some time to develop and execute - perhaps we will explore this in the future. If you have any ideas with how we might accomplish this, please reach out!

From this effort, we managed to get a total of 644,140 samples, split between 613,879 samples of malware and 30,261 samples of goodware.

Malware vs Benign Distribution

Exploration

So now we’ve shown how we collated and parsed our datasets, let’s step through what we found when we parsed them, and what our initial thoughts and theories were as we stepped through the datasets.

Architecture

The easiest place to start is with architecture. Let's take a look at how our dataset splits between x64 and x86 for both malware and goodware.

First, malware:

And goodware:

Malware Architecture

Goodware Architecture

Interestingly, for malware we have 613,878 samples as x64 and 1 unknown. Whereas, with goodware, it’s a bit more evenly split with 19,880 x64, and 10,380 x86 samples. As an anecdotal note, this matched our feelings as authors, but the totality of the split was a surprise. From a development perspective, watching the lowest common denominator of supported languages slowly creep up has been as relieving as it has been useful (here’s looking at you, .NET Core) and architecturally x64 has been around for a fair while but even so, it’s a surprise to see such a definitive absence of x86 samples. We can’t definitively rule out odd behaviours with how samples are submitted to datasets with this one but frankly, we’re not too sure but we are suspicious.

Sections

Using the PE Format debug page as a basid for officially implemented section names, we can directly compare the size and entropy of like sections, as well as use the section names as a list to check for unofficial and/or unconventional sections.

If we take all the sections from the data and put it into a JSON object where each section has a list of entropy and size values, we can then use those lists to create some averages. The table below summarises that data.

Firstly, the .text section, which typically contains executable code, shows notably higher average entropy in malware (6.26) than in goodware (5.99), which we would theorise is due to heavier obfuscation and/or packing in malicious binaries. Additionally, while malware .text sections are more numerous, goodware versions are on average larger, possibly reflecting more extensive legitimate codebases.

The .rsrc section is substantially larger and more frequent in malware. Its higher average entropy in malware (4.56 vs. 3.75) implies frequent misuse for embedding obfuscated or encrypted payloads. Conversely, .rdata, usually reserved for read-only data, shows a reversed trend: goodware has higher average entropy and larger size, potentially due to richer embedded constants or resources.

Goodware exhibits minimal use of some sections entirely absent in malware, such as .srdata (1 instance only) and .sxdata, although their low counts limit statistical interpretation.

Finally, the .debug section is disproportionately large in malware despite being rare, with an average size exceeding 1MB. This is unusual since debug symbols are generally stripped in production binaries; it may reflect intentional misuse or misclassification.

Overall, differences in section entropy and size offer useful signals for distinguishing malware from goodware, especially when considering packing behaviours and section utilization patterns. Reviewing this with our developer hats on, we couldn’t help but think this was due to malware authors pulling from a smaller pool of knowledge and online examples, potentially leading to less variation than goodware. For example, consider how many pentesters you know who use the ippsec nmap flag order.

Let's now do the inverse, taking this section names and only looking at the ones which we do not recognise - that gives us the following:

Malware samples exhibit a high prevalence of non-standard sections like UPX1, UPX0, and UPX2, which are indicative of UPX-packed executables. UPX1 shows very high entropy (6.40) which is consistent with packed files. Other sections like .bak and .NewSec suggest the use of injected or renamed sections, with .bak showing suspiciously high entropy (7.02), further hinting at obfuscation or polymorphism.

Conversely, goodware exhibits fewer and more consistent non-standard sections with lower entropy and smaller sizes, such as .buildid, and .00cfg. These sections are generally used for benign metadata or runtime configurations. The limited variation and low entropy in goodware indicate transparency and predictability, in contrast to the diverse and entropy-heavy nature of malware sections. Overall, the data underscores entropy and anomalous section naming as strong indicators for distinguishing malicious binaries from legitimate ones.

File Size

File size is another interesting component as this is often associated with both evasion and detection. We've seen bloaters increase the size of a file so that vendors don't scan it, we've seen tiny executables to have nothing to scan, etc. lets do some analysis on the dataset and see what we have.

Malware exhibits a broader and heavier size distribution than goodware. While 59.3% of malware samples fall under 1MB, a significant portion (35.8%) resides between 1--5MB, with outliers reaching up to 30MB. This suggests two common malware strategies:

1. minimalist payloads designed to evade signature-based detection by reducing surface area

2. bloating techniques to surpass size thresholds of certain scanners that ignore large files due to resource constraints.

Goodware, on the other hand, overwhelmingly clusters under 1MB (86.3%), with a median size of only 84.39KB. This consistency reflects structured software development practices and minimal code bloat, contrasting sharply with malware\'s diversity. Interestingly, both categories show negligible presence above 50MB, indicating such large binaries are rare and not typically used for either benign or malicious purposes. However, protecting against malware is not as simple as just blocking files that are over 50MB. Looking around our machines, we can see benign file sizes way above that (such as games, installers, and other trusted files) that shouldn’t be automatically deleted. Conversely, many libraries which we employ maliciously are above the minimum threshold, indicating that file size alone would not form a reliable rule, but could certainly contribute towards flagging in combination with other indicators.

Malware's higher standard deviation (2.41MB vs. 2.37MB) and median-to-mean size gap (688KB vs. 1.45MB) imply a right-skewed distribution influenced by large, bloated files. These may include packers, resource-stuffed payloads, or multi-stage droppers. Conversely, goodware's smaller gap between median and mean, coupled with a lower mean, reflects tighter control over distribution and fewer anomalies.

Whats interesting here is the obvious usecase between goodware and malware. Whilst goodware aims to avhieve so many different tasks from calculator apps to task managers, malware has a much stricter scope and often is trying to smuggle more malware in. this is a key limitation to our dataset, we do not have the malware family. We cant tell, easily, if a sample is ransomware, a dropper, or an implant itself.

Overall, malware's file size variability underscores its adaptive nature oscillating between stealth and excess while goodware trends show consistency, which can inform baseline profiling for anomaly detection in static analysis pipelines.

Entropy

Another highly discussed component is entropy, specifically Shannon entropy. This is used to measure randomness across data and with malware, encryption is used to protect more stuff the bad guys don't want you to see. For example, consider an executable that wants to execute shellcode to drop an implant... that shellcode will most likely be encrypted.

Below are some tables showing a summary of entropy across the files:

Malware tends to have higher entropy values on average (mean: 6.37) compared to goodware (mean: 6.10), and this difference becomes more apparent when examining the distribution across entropy ranges.

53.4% of malware samples fall within the 6.5 to 8 entropy range, suggesting that over half of the dataset uses some level of obfuscation or encrypted sections. In contrast, goodware peaks between 6.0 and 6.5 (40.6%) and sharply declines in the higher entropy bins, with only 5% of goodware exceeding 7.0 in entropy.

This pattern supports the hypothesis that high entropy can be a strong indicator of malicious behaviour. However, entropy alone is not a perfect predictor. For example, some benign software (e.g., installers, compressed executables, or cryptographic tools) may naturally exhibit higher entropy. The relatively lower standard deviation in goodware (0.7566) versus malware (1.3545) also indicates that benign files are more consistently structured and less likely to include large, encrypted blobs or random-looking data.

Imports

Next, the imports. In Categorising DLL Exports with an LLM, mez0 took Windows API function and used an LLM to categorise functions from MSDN into groups - the csv mapping file can be found here: https://gist.github.com/mez-0/833314d8e920a17aa3ca703eabbfa4a5

Looking at VirtualAlloc for example, this function is exported from Kernel32.dll and was categorised as such:

By reviewing imports, we can get a gist of what the file may get up to. The issue here is that there are a few ways to mask this and get the function address and call it dynamically, that is not what we are concerned with here. Below is an example of how imports look in Chrome:

For each sample, we can count how many times an import occurs to get an idea for how common individual calls. With that, we can cross-reference the goodware and look at any key differences.

Malware functions:

A key observation is that malware samples heavily utilize core Windows kernel functions related to process and file manipulation. Functions such as kernel32.dll!ExitProcess (63.66%), GetModuleHandleA (49.64%), and WriteFile (48.67%) are among the most frequently observed across malware samples. This suggests a reliance on programmatic control of execution flow and I/O operations. Functions like LoadLibraryA and GetProcAddress are also notable for their roles in dynamic code loading and function resolution common techniques used in malware to evade static analysis and enable modular payload delivery.

Goodware functions:

In contrast, goodware samples more frequently invoke system introspection and diagnostic functions, including GetSystemTimeAsFileTime (51.36%) and QueryPerformanceCounter (51.14%). These functions are typically used for performance monitoring, logging, and exception handling behaviours expected in legitimate software. The frequent appearance of exception-related functions such as SetUnhandledExceptionFilter and RtlVirtualUnwind further supports the notion that goodware tends to incorporate robust error-handling mechanisms, likely due to higher development standards and testing.

The shared set of functions, such as CloseHandle, GetLastError, and Sleep, appear in both malware and goodware but with significantly higher frequency and volume in malware. This reflects the general-purpose nature of these APIs; they are essential for both benign and malicious applications. However, the disparity in usage frequency may reflect differing operational intensities or structural design. For instance, the disproportionately high number of calls to GetProcAddress in malware (401,545 calls vs. 11,004 in goodware) reinforces the view that dynamic behaviour is a hallmark of malicious code.

Overall, the data suggests that while some API usage is unavoidable and overlaps between goodware and malware, distinct patterns exist in how these functions are employed. Malware tends to leverage APIs that facilitate code injection, persistence, and stealth, whereas goodware tends to use APIs aligned with reliability and diagnostics. This supports the use of API call profiling as a viable feature for malware detection and behavioural classification, provided it accounts for contextual usage and not merely presence or frequency.

Manifest

The manifest section of a file contains the description, trademark, etc. This can be seen under the "Details" tab of any file\'s properties:

Looking at the dataset, 39.44% of malware binaries (242,122/613,879) have manifests. Compared to 78.94% (23,888/30,261) of goodware. This kind of makes sense... software being provided by legitimate companies will fill this out, why wouldn't you? Whereas in malware, traditionally, theres no point - unless you're trying to blend in.

Goodware consistently demonstrates higher metadata completeness, with fields such as file_version, product_version, product_name, and company_name appearing in over 75% of samples. In contrast, these same fields are present in only 26 - 39% of malware samples. This discrepancy may reflect the priorities of legitimate software developers, who are typically subject to compliance, version control, and branding requirements factors largely absent in malicious development practices. Moreover, the near absence of fields like original_file_name in both datasets (0.01% in malware, 0.07% in goodware) suggests limited diagnostic value for certain metadata attributes. Interestingly, while fields such as comments and assembly_version are present in a minority of goodware samples, their even lower presence in malware (2--9%) implies that attackers tend to omit non-functional descriptive metadata, possibly to reduce file size or avoid revealing development artefacts. This divergence in metadata usage may offer a viable heuristic for automated malware classification, though care must be taken to account for adversarial adaptation, where malware authors may spoof or mimic goodware metadata to evade detection.

bce48fd0850969f93da9e85a0c65f74b10d85aee68fdf6fe0e65d881f52f69ad was found to contain a manifest, if we dump it, we can see what it contains:

Alternatively, we can look up 70bc155082505cdc0ec08428b39fe03212ff06653549f36cd8a9577202fc20a5 and see how it should be written:

It seems kind of obvious here, but this kind of data should probably be filled out. Granted, it won't secure an implant landing, but it's one less weird thing. So, to conclude, this is basically no effort to add, but it is worth it.

Toolchains

With all of that said and done, none of it would exist without the toolchains. This covers things such as:

• linker

• compiler

• packers

• etc

The best method for populating this, in our experience, is Detect-it-Easy. Firing it up and pointing it to a sample, we can see a ton of information.

There is also a python binding by elastic. However, it took an incredible amount of time over the sample count and ended up hanging. Effort was made to introduce a timeout wrapper on the python function, but it was still slow. Instead, a .NET application was built to async parse all samples - this worked perfectly fine and took 3 hours to parse all samples. The main command to do this from a CLI is:

diec.exe E:\dataset\malware\ -g -j -U -d --heuristicscan

The output from the .NET wrapper:
DIEC Scan Summary - 09/06/2025 01:48:39
==================================================
Total files: 616,524
Successful: 118,415
Failed: 498,109
Success rate: 19.2%
Top error types:
Process timeout: 498,109 files

Let's look through the data from linker, compiler, packer, and protector.

Linker

Analysing the data, we identified that 107,234 samples were able to have their linker identified. This gives us a percentage of 17.46% of samples we collected, were identifiable. Below is the full output:

Microsoft Linker is prevalent in both categories. This makes sense, the Visual Studio IDE is easy to work with, so why wouldn't most things in general be built with it? Turbo Linker, Polink, LCC Linker, Watcom Linker, GoLink, and TCC Linker are almost exclusively associated with malware (98.25%--100%), indicating these tools are either niche or specifically favoured by malware developers, possibly due to their flexibility or obscurity. Conversely, PowerBASIC and Borland TLINK are used only in goodware (100%), suggesting they are either outdated or tightly controlled, limiting their use in malicious contexts. GNU Linker stands out with a higher goodware percentage (61.18%) than malware (38.82%), likely since its easy to get access to, automate, and is seen in a lot of GitHub projects.

Compiler

Moving onto Compilers, we start to see even more variety.

Microsoft Visual C/C++ is prevalent in both (81.01% malware, 18.99% goodware). Again, this shows that it has widespread adoption and makes it a common choice for ~malicious~ developers. Tools like MASM, Microsoft Visual Basic, FASM, LCC-Win32, Virtual Pascal, Visual Objects, Intel C/C++ Compiler, Tiny C, and Go are exclusively or nearly exclusively (100% or near 100%) used in malware. However, this will be purely a dataset issue and perhaps speaks to the age of the samples obtained. This then appears again with Visual Basic (99.93% malware) and Borland Delphi (99.09% malware). MinGW (60.92% malware, 39.08% goodware) is often wrapped in open-source tooling, so this is no surprise.

Packer

Packers are used to protect other files by encrypted and/or compressing them into another file. Then, on execution, the original file is fully or partially decrypted/decompressed and loaded into memory. Below is the data we gathered.

What is interesting is that near 100% of this is malware usage. UPX (11,774 samples, 99.86% malware), Petite (2,603 samples, 98.42% malware), and MPRESS (2,032 samples, 99.95% malware) dominate in sample size. As much as genuine developers want to protect their work, malware authors are slightly more motivated to do so if they are trying to drop an implant or ransomware, for example.

Moreover, py2exe stands out with 78.57% goodware (11 of 14 samples), suggesting its use in legitimate Python-based software development, though its small sample size limits broader conclusions. This would make sense as python code is trivial to read and understand, py2exe whilst not a direct packer, it has the same impact. Other tools, like DxPack (1,632 samples) and NeoLite (923 samples), report 100% malware, but smaller sample sizes (e.g., PyInstaller, 1 sample) reduce the reliability of these percentages.

It's worth noting again, though, the limitations of the datasets which we have here. It was a lot easier to get malware then it was goodware, so the sample size can be more granular.

Protector

If you consider a packer to be something to package up and protect a file, protectors aim to obfusctte the binary itself. Take VMProtect for example, which offers the following list of capabilities:

• Obfuscation methods

• Mutation

• Virtualization

• Ultra (mutation-virtualization)

• Protection options

• Memory protection

• Import protection

• Resource protection

• Packing

• Debuger detection

• Virtual tools detection

• Additional features

• Console version

• Watermarks

• Script language

• Licensing system

• Activation system

• Virtual files

A direct quote:

Secure your code against reverse engineering, analyzing, and cracking. Use the advantage of code virtualization, which executes virtualized fragments of code on several virtual machines embedded into the protected application.

Below is our analysis.

VMProtect (619 samples, 99.52% malware) leads in sample size, followed by .NET Reactor (181 samples, 83.98% malware) and Confuser (156 samples, 100% malware). Only five tools - VMProtect (0.48% goodware), .NET Reactor (16.02%), Smart Assembly (14.93%), Crypto Obfuscator (6.52%), and Dotfuscator (7.69%) report any goodware, but their goodware percentages are low, and sample sizes vary significantly.

Similarly, as Packers, we see a ton of malware usage here. .NET Reactor and Smart Assembly, with notable goodware percentages, suggest some legitimate use, possibly in .NET-based development, but their malware dominance (83.98% and 85.07%) remains concerning. Tools like Confuser and SoftSentry, with moderate sample sizes and 100% malware, are high-risk candidates for security monitoring.

Code Signing

To try and validity to a file, code signing is used. We can see this with the default files included by Microsoft on Windows - consider NTDLL.DLL:

LolCerts was created to try and keep track of certificates that had been leaked, for example the MSI Certificate: MSI Data Breach: Private Code Signing Keys Leaked on the Dark Web

By code signing malware, the operators are hoping that vendors will allow the binary. Whilst writing this blog, we tried to find some official documentation pointing to code signing certificates having a positive impact to evasion, but the only thing we have to reference is experience of "this has worked for us". Below is a count of all the CA's we managed to parse:

As interesting as that is, it's more of a FYI. Here is the meat and potatoes: of the 613,879 samples of malware, we found that 568,456 (92.61%) of the samples were unsigned, and 45,423 (7.39%) of the samples were signed. Conversely, of the 30,261 goodware samples, 9,490 (31.36%) were unsigned and 20,771 (68.64%) were signed. So, goodware tends to be signed often whereas for malware the opposite is the apparent case. So again, we cannot definitively prove that signing will make a difference or that it would be the sole reason that you can blend in, but it does act as yet another thing that will help a binary blend in more. However, getting a code signing certificate is difficult; while ADCS can give them out, we don't see any research showing the impact of signing a file with an internal ADCS code-sign certificate, whereas compromised certificates and known-bad issues would clearly not be a good inclusion. The thing to note is that even legitimate services use commercially available developer certificates which are add to the air of authenticity and further divorce a binary from the malware aesthetic.

Conclusion

The dataset was a bit of a chore to obtain, across both goodware and malware but we did what we could, and we are always looking to still ingest more. Of the two, malware is easier to come by, than goodware and less problematic to distribute (from a licensing perspective), but equally may be somewhat modified depending on the state and source.

Our main initial takeaway from this would be that if you act like a legitimate developer, you look like one too.

That key differentiation between the two datapoints makes for an interesting comparison. However, we noted that for a binary to best blend in, it’s going to sound obvious, but develop more like developers. Malware built and setup properly with tools like Visual Studio with all the right resources, looks a lot more genuine. Trying to blend into the grey between malware and goodware is preferable. If any detection logic is written purely on “if no manifest”, then the rule deserves to be broken and hopefully vendors writing poor logic like that can learn something. As with all our blogs, we don’t want to give you the surefire results to build better malware, if you read the blog end to end and are capable, you will know what to improve.