Forensic repository for both humans and machines

foorep is an open source forensics/malware repository written in Python that is easy to install, scales if you need it and provides interfaces for both humans and machines.

The "Hello World" for #dfir people today seems to be creating their own malware repository. I needed to learn mongoDB and thought that yet another foo repository couldn't hurt.. So after a couple of late nights, we have this:

Features at a glance

• CLI
• WebUI
• RESTful API
• Python library
• Search
• Simple but powerful annotations
• Plugin system for static analysis

Easy to install
If you are using Ubuntu it is a 2-step experience (MacOSX is untested, but will probably fail. I will fix this). First you need a mongodb database. If you have one running you can use that, if you don't you need to install one. By default mongodb will listen on localhost with no authentication. If this is something you can live with, you are good to go.

Note: This is a _very_ early release/hack done on my non-existing spare time, so there are rough edges all over. Drop me a line if something isn't working for you or even better, send a bug report!

1) $ apt-get install mongodb
2) $ pip install foorep

Then try it out:
$ foorep add /path/to/malware
$ foorep list

Or start the built in webserver and use the webUI
$ foorepd

Then open your browser and go to http://localhost:4780

Annotations
foorep has a very flexible way to add annotations to your samples. You are able to tag and comment, but it doesn't stop there. Annotations in foorep are simple JSON objects that we embed in the metadata document. This makes it possible to invent new types of annotations as we go.

Plugins
In order to do something more than just store the files, I created a simple plugin system that automatically do some static analysis on the malware being uploaded. The plugin will recieve a filehandler or a path to a file and it should return a foorep annotation (python dictionary).
Implemented plugins:

• PEfile - if the file is a PE file, pick it apart and do some analysis
• exif - if the file is a image with exif data in it, annotate the sample with the exif tags (currantly you need to install python-exiv2, apt-get install python-pyexiv2)

I am planning on creating plugins for Yara, Virustotal or Team Cymru Malware Repository to name a few.

Sharing
This is something that will be implemented in the future. But I will integrate fordrop in foorep to make it possible to share and federate the samples in your repository with others. Making it a decentralized and distributed malware repository. We will use XMPP pubsub for this. See fordrop for more information.

The use of mongoDB

MongoDB (from "humongous") is a scalable, high-performance, open source NoSQL database. 

MongoDB is a document oriented database were you store JSON documents. This is a perfect fit for a malware repository, and if I can do without SQL schemas I'm happy!
foorep stores metadata from malware samples as JSON documents, making them searchable. We also store the raw files in mongoDB, using GridFS.

Use in other python programs

import foorep
my_repo = foorep.Repository()
my_repo.insert('/path/to/malware')



RESTful API
If you are a machine and want to consume and produce data in foorep but you are not a python, you can use the RESTful HTTP based API. In this initial release the API is just read-only, but I plan on implementing the rest in the next release. I will write more on this soon.

curl http://127.0.0.1:4780/api/v1/file/d02aa8cb63084b6cbdfab9cb68343cf7

Hacking

Implemented in python with some test coverage (adding more tests is a priority as always). If you would like to help out adding features or crushing bugs the source is available over at github.

Open source

This project is released as open source using a 2-clause BSD license. This basically means that you can take it and do whatever you like with it. Please do.