…and processing samples faster than ever. We’re up on the migrated server with a few gotchyas along the way, but the end result is Minotaur sped through over 3,000 samples yesterday while catching up from the downtime.

A while back I wrote a blog post on all the different virtualization technologies in us in Minotaur. This weekend, I’ve started consolidating the backend of Minotaur on Citrix’s XenServer. As you can guess, this is a large undertaking and while I expected the downtime to just last the weekend, it appears the controller may be down for several more days for conversion and migration. All the data on the frontend website should function as usual with the exception of the DNS system.

During the downtime I’ve taken the opportunity to simplify the system. I still had code running from the NUMAN days and still was running each sample through zerowine but not really doing anything with the results. The backend has been significantly reduced and optimized and should make my life of day-to-day administration (feeding the bull) much easier as well as make the system more reliable as a whole.

Minotaur has gotten a few new upgrades. Most notably on the web frontend is the addition of two new sections in the sample reports. Now, each sample’s traffic capture will be further analyzed for the URIs accessed and all DNS requests and responses.

One hope is that this analysis will provide for a rich dataset for further analytics based on URI patterns, and DNS anomalies.

As an example, the following backchannel requests popped up from sample 6da34a083feef6f9553e492e10537ca5:

Oh how I wish they all were so easy they put “logdata=infected” and “logdata=Executed%20payload” in the URI.

We’ve had quite a few upgrades to minotaur this weekend. First, and most noticeable on the homepage is a display of recently analyzed network communications of malicious samples. If you click on a particularly interesting map, it will take you to the report for that sample.

Next, you can now search for samples using the MD5, SHA1 or SHA256 hashes which will hopefully improve the usefulness of the search system.

Lastly, the backend that generates the videos, screenshots, tcpmaps and dynamic analyses had a major flaw in the way files were copied into the environment that was preventing some of the samples from having full data. This bug has been fixed which should lead to more consistent level of analysis of EXE files.

Hopefully many more updates on the way.

The anti-malware dns aggregator is back online on a completely new backend and statistics should be rolling in very shortly. I’m running the entire domain database through the new system, so there will be a large spike in detections over the next few days. Please, if you know of any more services we can add to this aggregator, let me know.

Minotaur uses several visualization technologies from different vendors. In case this is useful for others setting up their own dynamic analysis environments, I thought it would be a good idea to chronicle some of our efforts here.

On our workstations we run a combination of VMWare, VirtualBox, and Parallels. Depending on the specific needs and the analyst. I think on the workstation side, these solutions are so similar that we don’t really have a favorite.

Our dynamic analysis is split into two parts, one side runs samples through ZeroWine on 24 VMs running in Proxmox, and the main Windows runtime analysis is performed in CuckooBox, running in VirtualBox on a dedicated server.

Our primary infrastructure is run on two Proxmox nodes. This infrastructure covers the collectors, databases, analyst workstations VMs, and the data environments. Proxmox allows Minotaur to have great VM density and allows us to run more than 60 active virtual machines on two off-the-shelf hardware nodes. Until very recently, all of the storage was local on the servers, so even though Proxmox features live migration capabilities, we were unable to use this in-house.

Enter our new storage array from Synology, the DS1812+. The Synology box gives us the ability to centralize our storage using iSCSI LUNs while still being a pretty affordable solution.

So the idea was I would just migrate some of the Proxmox nodes to central storage so that we would have better availability and load balancing capabilities. But then I became aware of some of Citrix’s current offerings, namely XenServer and XenDesktop, including the free versions of each. I especially took notice of the management capabilities of the systems using windows consoles. I can also see how direct citrix access to the workstations would be of great value to our analysts. So I decided to start a pilot. I’ve since offloaded two of our physical servers that were running internal infrastructure to the Synology box and loaded Xenserver on the first and will load the second one up as a windows server to run the XenDesktop components in the near future.

So far the setup was very easy after getting used to the different conventions. Setting up all the VLANs was easy, and setting up the iSCSI LUN was a snap as well. I decided to try running a conversion for the first box, so I converted a linuxMint VM from VMWare Workstation. This did not go so well, but I would not use a converted machine for production anyway, always best to build a clean image on the architecture. I would also guess that a windows workstation would have gone smoother, as that’s usually what conversion tools are catered toward.

Having installed the first sever VM and setting up the guest tools, everything looks to be running pretty smoothly now.

Minotaur is up for a rewrite of the backend components as we move to the current versions of cuckoobox that include MAEC reporting (among other significant improvements) which will hopefully increase Minotaur’s benefit to the community. With major components being upgraded and re-tooled for this environment, now seemed like a good time to review our options for the entire infrastructure. It will be a while before any decisions are made and we know what fits our needs best but I do like playing with new technologies.

Minotaur has added several new features of the last couple weeks. Most of these features have to do with backchannels. Backchannels are network communications that malware uses to “call home”. These communications can be anything from retrieving new commands and configurations to simple lookups of public information from public sources. Minotaur keeps track of all communications that take place during the execution of malware in the sandbox. It then correlates all of these communications with each other and produces a list of the top destinations of this traffic. Minotaur also produces a map of all communications that take place during the execution of the sample. Below is an example of such a map.

These capabilities are very much still a work in progress. We hope to soon provide much more information about each IP address and each communication. In the meantime we are building a database of all known back channels that Minotaur observes. The first fruits of this database can be seen in the link below.

BackChannels

…what’s 15,000 pictures worth?

Yup, minotaur now saves a video of each (relevant) sample processed via it’s cuckoo VMs.

What does it look like? WHy not check out a few samples with videos:

Fake AV
Hupigon

And for some old school mayhem:
Joke.Program

The system is automatically recording new samples as they come in as well as back-filling samples as it has time.

If it’s true a picture is worth a thousand words, then things just got a lot more interesting here…

That’s right… Minotaur now has screenshots taken at intervals during the execution of the malware in one of our sandbox systems (cuckoo)

Not all samples have screenshots. For some examples, try:

• 149c6c045b0b50dd158f160af75ded60
• a1707abae656968dc069a483ec848bd6
• 58d1c30452243b2a0682a7f5ff9d1fd8
• 918db0dbacb499775f12fa89cae0f0a9
• c2854eec4e766ec33c223bbae3a43819
• ce9787ed7a281f28c57b2f7fabd7d7c3

Once we catch up on the backlog, the system should add these screenshots as each sample is analyzed (exe only for now, PDF next).

Because cuckoo can run these dynamic analysis routines faster than the other sandbox environments we’ve built, it is becoming an integral part of the Minotaur platform. We are just working on scaling it up to what minotaur needs now.

Minotaur now presents all stats on every malware family we track here.

The list page presents little maps of the average location of our detections of each malware family. While still in the very early stages of developing these tools, I have noticed the vast majority of these maps center in on europe. At this time, I believe this is due to that region being the intersection of all the points from otherwise very diverse geographic locations, and is not indicative of raised activity in europe.

Clicking on a family name will take you to our detailed statistics for that malware family, including a map of the most recently observed distribution servers. There is also a list of the actual samples here. Clicking on a sample will bring you to our detailed report on that particular sample.

This page will show you everything we know about a particular sample, including filetype probabilities, vendor concurrence, detections by all vendor engines, and links to outside information. In the near future, upgrades will allow you to pull the raw data reports from our tools for each sample.

We’ve also been busy integrating our different toolsets. For instance, in the detailed malware sample reports, near the bottom we have integrated our anti-malware DNS system’s known info for the originating site’s domain:

And very importantly, we are working on integrating a discussion engine into every page for every family, every sample, every category, everything. Feel free to leave a comment on any object you want, as it builds our community and could help out the malware research community as a whole by sharing what we know with each other.