28 May 2013

Noriben Version 1.2 released

In a mad rush of programming while on a plane to BSidesNOLA, and during the conference, I completed a large number of updates, requests, and demands for Noriben.

As a basic malware analysis sandbox, Noriben was already doing a great job in helping people analyze malware more quickly and efficiently. However, it had its bugs that hurt a few outlier cases. Using submitted feedback (through email, twitter, oral, and death threats) I believe that the major issues have been fixed and that the most-needed features have been added.

New Improvements:

  • Timeline support -- Noriben now automatically generates a "_timeline.csv" report that notes all activity in chronological order, with fields for local time and a grouping category. Feedback is welcome for ways to improve this output. For example:
8:16:19,Network,UDP Send,hehda.exe,2520,
8:16:19,Registry,RegDeleteValue,hehda.exe,2520,HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run\Windows Defender
  • Tracks registry deletion attempts -- Older versions only tracked successful deletions to the registry, assuming that the keys and values existed. Now, it logs even when the keys don't exist. This opened up a large amount of data that was previously filtered out, such as ZeroAccess removing the services for Windows Defender and Microsoft Update (which weren't running on my analysis VM).
  • Large CSV support -- The old versions of Noriben read the entire procmon CSV into memory and then parsed them for results. This created numerous Out of Memory issues with very large sample files. The new version fixes this by only reading in the data one line at a time.
  • Parse Procmon PMLs -- PML files are the binary database used to store the native events during capture. These are converted to CSVs during runtime, but a number of users have years worth of saved PMLs for previous malware samples. Now, Noriben can just parse an existing PML without having to re-run the malware.
  • Alternate Filter files -- Previous versions of Noriben required that you use one filter file, ProcmonConfiguration.PMC, to store your filters. This created issues for users who maintained multiple filters. A new command line option has been added to specify a filter file. This can be used in conjunction with the "-p" PML parsing option to rescan an existing PML with new filters.
  • Global Blacklists -- There was a need for a global blacklist, where items contained in it (namely executables) would be blocked from all blacklists. That allows for a blacklisted item that doesn't have to be manually added to each and every list. 
  • Error Logging -- In a few unusual cases, Noriben fails to parse an event item from the CSV. While Noriben contains proper error handling to catch these issues, it just drops them and moves on. As these events may contain important items, they are now stored in raw at the end of the Noriben text report for manual analysis. If something looks amiss, and they are extremely important items, the list can be emailed to me for analysis and better handling in future versions.
  • Compartmentalized sections -- This is mostly a back-end, minor feature, All events are now grouped into separate lists for Process, File, Registry, and Network. 

General fixes:

  • Changed "open file" command for Mac OS X to 'open'. OS X is tagged as 'posix'. This allows for Noriben to parse files from a Mac interface, but this is not recommended. Parsing files on a system other than the infected means that system environment variables, such as %UserProfile%, will not be identified correctly.
Noriben has changed its command line arguments, dropped the '-r' (rescan CSV) and introduced more specific arguments per each file type, '-c' (CSV), '-p' (PML), and '-f' (filter):

--===[ Noriben v1.2 ]===--
--===[   @bbaskin   ]===--

usage: Noriben.py [-h] [-c CSV] [-p PML] [-f FILTER] [-d]

optional arguments:
  -h, --help                   show this help message and exit
  -c CSV, --csv CSV            Re-analyze an existing Noriben CSV file [input file]
  -p PML, --pml PML            Re-analyze an existing Noriben PML file [input file]
  -f FILTER, --filter FILTER   Alternate Procmon Filter PMC [input file]
  -d                           Enable debug tracebacks