tag:blogger.com,1999:blog-24569105082355773402024-03-16T23:53:30.626-07:00Another Forensics BlogDigital Forensics and Incident Response Research,Python Scripts and Musings Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.comBlogger38125tag:blogger.com,1999:blog-2456910508235577340.post-24453383360147711312020-02-27T10:21:00.000-07:002020-02-28T17:32:13.658-07:00Detecting Lateral Movement with WinSCP<div class="separator" style="clear: both; text-align: center;">
</div>
RDP is a common way for an attacker to move laterally within an environment. Forensically, when an attacker uses RDP we can use artifacts such as shellbags, link files and jumplists on the remote system to see what was accessed while the attacker was RDPed into the system.<br />
<br />
Another way an attacker can access a system remotely is to use a program called WinSCP. Using<br />
WinSCP, they can browse folders and files on a remote system, copy folder and files back to the system they are currently on, and even search the remote system for files!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB9givHtbzp8u50yEwC8gbom46qAZB9U9F-_Jh3KcR1vP9spuyWpl33KJcKO040N5FnnIdx69YQEJFsqvUJkxJI3_C-6U-zKY9MzVP7G38vnCwGpARBfr2l4fWD3nS9vqVcEHscMWsy_wY/s1600/winscp.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="648" data-original-width="971" height="426" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB9givHtbzp8u50yEwC8gbom46qAZB9U9F-_Jh3KcR1vP9spuyWpl33KJcKO040N5FnnIdx69YQEJFsqvUJkxJI3_C-6U-zKY9MzVP7G38vnCwGpARBfr2l4fWD3nS9vqVcEHscMWsy_wY/s640/winscp.png" width="640" /></a></div>
<br />
<br />
The scenario I am going to focus on here is one where the attacker has already compromised a system on the network, and is using WinSCP to browse to other computers on the same network. In this case, they could browse to HR systems looking for tax information, Severs looking for databases or Workstations looking for IP data. (Note - TLDR at the bottom)<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEM5cWfb_y-2Z5yjg5bvBOWgCHVuXenN2ytIxaqhMJxYsU6A8apOpVfqnTKIF56ev6gi7L5XtoKKiUb_dCgLSQ97VhwCX994Aln0ys9PJ9PGQ93XxypgXvN0PnxDf3o0h244hVdvW7v57X/s1600/winscp_network.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="645" data-original-width="859" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjEM5cWfb_y-2Z5yjg5bvBOWgCHVuXenN2ytIxaqhMJxYsU6A8apOpVfqnTKIF56ev6gi7L5XtoKKiUb_dCgLSQ97VhwCX994Aln0ys9PJ9PGQ93XxypgXvN0PnxDf3o0h244hVdvW7v57X/s320/winscp_network.png" width="320" /></a></div>
<br />
Because they are not using the Windows Explorer shell, this leaves very little artifacts on what they were doing on the remote system in comparison to RDP. Basically, they get a browse for free card. They can even open up remote documents from within a WinSCP text editor. <br />
<br />
An argument may be made that RDP is available by default on Windows systems while FTP/SSH is not. Well, guess what. Starting with Windows 10 1809 and Sever 2019 it is part of the "optional features " that can be easily installed on Windows. In fact, a simple PowerShell command can be used to install it. And, on top of that, it automatically creates a firewall rule and adds an SSH user. How thoughtful!<br />
<br />
<div class="code">
powershell Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0<br />
powershell Start-Service sshd</div>
<br />
And as a bonus, add the command to have the service start up automatically: <br />
<div class="code">
powershell Set-Service -Name sshd -StartupType 'Automatic'</div>
<br />
It is not uncommon for an attacker to follow the below steps once they have breached a network:<br />
<br />
1) Dump admin credentials<br />
2) Enumerate systems to get IP addresses/Hostnames<br />
3) Push out PowerShell scripts to all systems en-mass that do things like disable firewalls, install backdoors and disable antivirus.<br />
<br />
It's a simple task to add in one more command to install SSH and now boom - all these systems are now accessible to connect to using WinSCP.<br />
<br />
Oh - and did I mention that WinSCP comes with a portable version? The portable version makes it easy for an attacker to download and use. Many blog posts reference a registry key that contains settings for WinSCP. However, the portable version does not store settings there. <br />
<br />
So - now that we know WinSCP can be used in this manner, what artifacts can we find forensically to help determine what was done on both the "staging" system and the remote systems? I did some testing on some Windows 10 1909 machines to see what artifacts were left behind using the Portable version of WinSCP, v.5.17.<br />
<h2>
</h2>
<h3>
<u><b>WinSCP Client System Artifacts</b></u></h3>
<br />
Most of the artifacts related to WinSCP are going to be on the host where it was run. Running WinSCP generates many of the common artifacts seen with file execution: Prefetch, shimcache, amcache, userassist etc. However, the artifacts "for the win" will be the WinSCP.ini file and the SRUM database.<br />
<br />
<b>WinSCP.ini file</b><br />
<br />
WinSCP.ini is a text file that contains configuration settings. It will be located in the same directory as the WinSCP.exe file. At the end of a WinSCP session, the user is promted to save their workspace: <br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGTZ1R4lxYYrFvswI_vxuiQ8T77GI0j9OA_4p8mYeAuD2u9n1OJOLS9_UagTFz2ktTBU1mdpoGzk9MQqIdOtGE1QRjj9tw_NUe0N0PVvde-khUlAyMqlnT1FSZynwF70K20fhX9ulHzfyK/s1600/confirm_save.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="173" data-original-width="499" height="137" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGTZ1R4lxYYrFvswI_vxuiQ8T77GI0j9OA_4p8mYeAuD2u9n1OJOLS9_UagTFz2ktTBU1mdpoGzk9MQqIdOtGE1QRjj9tw_NUe0N0PVvde-khUlAyMqlnT1FSZynwF70K20fhX9ulHzfyK/s400/confirm_save.PNG" width="400" /></a></div>
<br />
<br />
<b>Even without saving the wokspace,</b> WinSCP saves valuable information in the WinSCP.ini file that can be useful to the investigation. This includes systems connected to,usernames, places on the local system where files were saved to from the remote system and the last path that was accessed on the local system. Examples of each of these configuration sections are below:<br />
<br />
<b><u>Systems connected to: </u></b><br />
<u><br /></u>
<u>[Configuration\CDCache]<br />ItSupport@169.254.249.229=412F433A2F55736572732F<SNIP><br />mdegrazia@169.254.171.129=412F433A2F55736572732F<SNIP><br />ITSupport@DESKTOP-PV2TN0G=412F433A2F55736572733D<SNIP></u><br />
<br />
<b><u>Folders where files have been saved:</u></b><br />
<b><br /></b>
[Configuration\History\LocalTarget]<br />
0=C:%5CUsers%5CCrashOveride%5CDocuments%5CExfil%5C*.*<br />
1=C:%5CUsers%5CCrashOveride%5CDocuments%5CSystem3%5C*.*<br />
<br />
<b><u>Last folder opened on the local system:</u></b><br />
<br />
[Configuration\Interface\Commander\LocalPanel]<br />
DirViewParams=0;1|150,1;70,1;120,1;150,1;55,0;55,0;@96|5;4;0;1;2;3<br />
StatusBar=1<br />
DriveView=0<br />
DriveViewHeight=100<br />
DriveViewHeightPixelsPerInch=96<br />
DriveViewWidth=100<br />
DriveViewWidthPixelsPerInch=96<br />
<b>LastPath=C:%5CUsers%5CCrashOveride%5CDocuments%5CExfil </b> <br />
<br />
<br />
If the session settings are saved, you get a bonus section called Sessions, with the saved session name. The default is "My Workspace" . This saves the last local directory and remote directory, along with a password. Check out https://github.com/winscp/winscp/blob/master/source/core/Security.cpp for information on the password encryption.<br />
<br />
<b>[Sessions\My%20Workspace/0000]</b><br />
HostName=169.254.44.249<br />
UserName=ITSupport<br />
LocalDirectory=C:%5CUsers%5Cmdegrazia%5CDocuments%5CSystem3%5CW2s<br />
RemoteDirectory=/C:/Users/Acid%20Burn/Documents/W2s<br />
IsWorkspace=1<br />
Password=A35C435B9556B1237C2DFE15080F2<TRUNCATED><br />
<br />
The WinSCP.ini file appears to be updated when the session closes. As such, using the last modified date of the WinSCP.ini file with a prefetch timestamp could give you an idea of how long the last session was.<br />
<br />
As you can see by the information above, looking at this .ini file can help an examiner determine what an attacker may have been browsing to on a remote system, and what they may have saved on the local system, even if it was subsequently deleted.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<u><b>SRUM Database </b> </u><br />
<br />
The SRUM database collects information every hour on network usage on a per application basis. To get an idea of how much data may have been copied/downloaded using WinSCP it can be an excellent resource. Parsing the SRUM database with <a href="https://github.com/MarkBaggett/srum-dump">SRUM Dump</a> by Mark Bagget shows that a high amount of data was transferred using WinSCP:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTs5WbwAX5gkct5AVy7jodLlg9Vnn7sGTCKhUPqgNs_5nPoy9IBKoaOudH5wwPt-5_9fcCjw_2vQKxcYvLCyjPNV_SqCTq_WZv8RA8krS8qd6EY6Opb8ZddCj8XAzoKn5pwKPCrk9pisYX/s1600/SRUM-Received.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="310" data-original-width="1383" height="142" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTs5WbwAX5gkct5AVy7jodLlg9Vnn7sGTCKhUPqgNs_5nPoy9IBKoaOudH5wwPt-5_9fcCjw_2vQKxcYvLCyjPNV_SqCTq_WZv8RA8krS8qd6EY6Opb8ZddCj8XAzoKn5pwKPCrk9pisYX/s640/SRUM-Received.PNG" width="640" /></a></div>
<br />
As demonstrated above, if you suspect WinSCP was used, parsing out the database can provide some details on how much data was transferred, what user account was associated with it, and the time frames that it occurred. Beautiful!<br />
<h3>
<u><br /></u></h3>
<h3>
<u>WinSCP Remote System Artifacts</u></h3>
There are several things you can look for on a remote system to determine if WinSCP was used to browse it: Event log entries, evidence of OpenSSH being installed and file system timestamps. Note - in my example and for my testing I installed OpenSSH which is part of Windows. WinSCP can use other FTP/SSH servers to connect to. Keep that in mind if you suspect WinSCP may have been used - your artifacts may vary.<br />
<br />
<h3>
OpenSSH artifacts</h3>
As mentioned previously, in order for WinSCP to connect to a system, an FTP or SSH sever must be running to accept the connection. Look for artifacts indicating these services exist. For OpenSSH, look for c:/Windows/System32/OpenSSH/sshd.exe, SSHD.exe prefetch files, and the sshd.exe service. Timestamps associated with these entries may help determine the first time the attacker used it to connect. When I installed OpenSSH, it also created a user account, which can be located in the SAM hive (shown here parsed with <a href="https://github.com/keydet89/RegRipper2.8">RegRipper)</a>:<br />
<br />
<b>Username : sshd [1003]</b><br />
SID : S-1-5-21-1445295406-4253784506-242647837-1003<br />
Full Name : sshd<br />
User Comment : <br />
Account Type : <br />
Account Created : Sun Feb 23 06:48:08 2020 Z<br />
Name : <br />
Last Login Date : Never<br />
Pwd Reset Date : Sun Feb 23 06:48:08 2020 Z<br />
Pwd Fail Date : Never<br />
Login Count : 0<br />
--> Password does not expire<br />
--> Normal user account<br />
<br />
<h3>
</h3>
<h3>
Event Log Entries</h3>
As expected there is an Event ID 4624 associated with the WinSCP client login. The login is a type 5 with the account name sshd_1860 and the domain of VIRTUAL USERS, and the process of sshd.exe:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9zlM2vmMAdcBUr4bqnh1Tw5qPcms1WXgxsM1kntGWMUxCfs-rlKIRZzyAU8wD2o7nNkQOZH_IC-2ubcr8GX8aHmSNBQB-xeDqfqZvxvexnZuVExVr_doLXOXDYcoAYTnYuNqSI-vA5p9k/s1600/4626_type_5_sshd.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="811" data-original-width="633" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9zlM2vmMAdcBUr4bqnh1Tw5qPcms1WXgxsM1kntGWMUxCfs-rlKIRZzyAU8wD2o7nNkQOZH_IC-2ubcr8GX8aHmSNBQB-xeDqfqZvxvexnZuVExVr_doLXOXDYcoAYTnYuNqSI-vA5p9k/s640/4626_type_5_sshd.png" width="498" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
This is followed by an entry in the OpenSSH Operational event log that records the connecting IP address and account used by WinSCP to connect:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTAgQwfMgrZ8skOYN1UU1lDQNdid8K3Zd31G3uP3pSJF0QNckmHA7ZJO6Nzu71lnFj3qCdy_LnHU5S6ddhbU9_NoEBq-nZVRyKxDZImEkHc9tyDj4cuZvgLfANltKYUr-cAuQHUbvIYn52/s1600/OpenSSH.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="453" data-original-width="643" height="450" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTAgQwfMgrZ8skOYN1UU1lDQNdid8K3Zd31G3uP3pSJF0QNckmHA7ZJO6Nzu71lnFj3qCdy_LnHU5S6ddhbU9_NoEBq-nZVRyKxDZImEkHc9tyDj4cuZvgLfANltKYUr-cAuQHUbvIYn52/s640/OpenSSH.png" width="640" /></a></div>
<br />
<b>File Timestamps </b><br />
<br />
Once logged in, the attacker can use WinSCP to effectively browse through folders, and even open up files via WinSCP leaving very little trace on the remote system. During testing, I noticed that an indication this was occurring was that accessed dates were changed on folders and files clicked on or copied. However, access dates are NOT a reliable artifact to use when drawing conclusions and must be used with other corroborating artifacts.<br />
<br />
Below is an example of files and folders that were copied. The "Teslacam" folder was copied, which results in the access dates of all the copied files to be updated on the remote system:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8mNarVW0MW9phyRCEMsP6Tnzzdmcpe9EdAfb6AVavTRkHXxNjzIJxPQ2KpmYl69ySoBJE8hhekw-O93jDRjmqdD0m1R-0gOGe9ZSRLdxOY3EoftqZT1KAT4pjIng9hh2239LPQY4JMT9b/s1600/file_copy.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="811" data-original-width="1600" height="324" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8mNarVW0MW9phyRCEMsP6Tnzzdmcpe9EdAfb6AVavTRkHXxNjzIJxPQ2KpmYl69ySoBJE8hhekw-O93jDRjmqdD0m1R-0gOGe9ZSRLdxOY3EoftqZT1KAT4pjIng9hh2239LPQY4JMT9b/s640/file_copy.PNG" width="640" /></a></div>
<br />
<b>SRUM Database</b><br />
Once again, the SRUM database really shines here to know if something is amiss. Looking at the SRUM database and sorting by "Bytes Sent" shows a large amount of data being sent during this time frame by the application sshd:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivAVcnq6DwGuktQtmSiGgwEKBk4uw3siTxs9ASmLh6x1O2e1YbC-AYYgX-puK4zKObSfr7cdiQRUWFN_A3OiWoCCMSbofwfxjYFX6kgtCpO1M6THeDX4SDVUW1AqwbD9U2vxjHoL4XUzIO/s1600/SRUM_Sent.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="306" data-original-width="964" height="202" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivAVcnq6DwGuktQtmSiGgwEKBk4uw3siTxs9ASmLh6x1O2e1YbC-AYYgX-puK4zKObSfr7cdiQRUWFN_A3OiWoCCMSbofwfxjYFX6kgtCpO1M6THeDX4SDVUW1AqwbD9U2vxjHoL4XUzIO/s640/SRUM_Sent.PNG" width="640" /></a></div>
<br />
<br />
<br />
<b><TLDR></b><br />
<br />
So, in summary. WinSCP can be used by attackers as an alternative to RDP. The use of WinSCP to access systems in an environment appear to leave a smaller footprint than using RDP. Now that SSH can be easily installed into Windows 10 and Windows Server 2019, I anticipate we may see WinSCP being used more in network breach cases to move around laterally within the environment. <br />
<br />
Look for the WinSCP.ini file on the host system and the SRUM datbase. For the remote system, look for Event IDs 4624 related to ssh clients/servers and Application logs for FTP/SSH severs. Check the SRUM database for data transfers related to ssh clients. Once a timeframe is known, check for large amounts of files that have last access timestamps in the same timeframe (but you know the drill with last access dates - be very careful using these)<br />
<br />
Sources/References:<br />
<br />
WinSCP: https://winscp.net/eng/index.php<br />
Installing OpenSSH on Windows: https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse<br />
SRUM dump : https://github.com/MarkBaggett/srum-dumpMari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com0tag:blogger.com,1999:blog-2456910508235577340.post-34393737917532138412019-08-21T13:44:00.002-07:002019-08-27T14:06:43.317-07:00Triage Collection and Timeline Generation with KAPE<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2-aXO_79-qV1H9zTPthcUMCMk0WeF17aiLc1wss17v-cukSRRoKWIMQvBsUI-kHmDr5yhiniICBLwLfEir1vh8Q5j01ZKgHdnOOYOQw8_V_G5ihKq3-hIIRYxB8cLx5w0XaAVsUtD8BLH/s1600/overview.PNG" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="369" data-original-width="683" height="172" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2-aXO_79-qV1H9zTPthcUMCMk0WeF17aiLc1wss17v-cukSRRoKWIMQvBsUI-kHmDr5yhiniICBLwLfEir1vh8Q5j01ZKgHdnOOYOQw8_V_G5ihKq3-hIIRYxB8cLx5w0XaAVsUtD8BLH/s320/overview.PNG" width="320" /></a></div>
As a follow up to my SANS <a href="https://www.sans.org/webcasts/triage-collection-timeline-analysis-kape-109840">webcast</a>, I wanted to post detailed instructions on how to use KAPE to collect triage data and generate a mini-timeline from the data collected. As much as I hate to say "push button forensics", once you get KAPE up and running, it really is only a matter of a couple of clicks and you are off to the races.<br />
<br />
I won't go into detail here on the benefits of collecting triage data or timelining (of which there are many!), but instead focus on how to set up KAPE to do it. If you would like more details on the above, please watch my webcast.<br />
<br />
To get the timelining to work in KAPE you will need to do three things to get it set up. These will each be detailed in this post:<br />
<br />
1) Download/Upgrade KAPE<br />
2) Grab the timeline Targets and Modules if you have an older version<br />
3) "Install" the executables called by the KAPE modules I wrote<br />
<br />
As KAPE gets updated, I expect step #2 to drop off as it will be rolled out with the newer versions.<br />
<br />
<h3>
KAPE Basics</h3>
<div>
<br /></div>
<div>
KAPE (Kroll Artifact Parser and Extractor) is a free tool written by Eric Zimmerman, and available for <a href="https://learn.duffandphelps.com/kape">download on the Kroll website</a>. From the website: </div>
<blockquote class="tr_bq">
"KAPE is a multi-function program that primarily: 1) collects files and 2) processes collected files with one or more programs. KAPE reads configuration files on the fly and based on their contents, collects and processes relevant files. This makes KAPE very extensible in that the program’s author does not need to be involved to add or expand functionality"</blockquote>
<div>
To this end, I have written a target that defines what files to collect to create a timeline, and about 20 modules that tell KAPE how to process the data - AKA - make a timeline.</div>
<div>
<br /></div>
<div>
In order to do this, you will need to grab the new target and timeline modules and the binary files that the modules call.</div>
<div>
<br /></div>
<h3>
Step 1 - Download/Update KAPE</h3>
<div>
If you don't have KAPE, download KAPE from <a href="https://learn.duffandphelps.com/kape">here</a>.</div>
<div>
<br /></div>
<div>
If you already have KAPE, you will need to have version 0.8.6.3 or greater. To update KAPE, run the Get-KAPEUpdate.ps1 PowerShell script in the root of the KAPE directory.</div>
<div>
<br /></div>
<h3>
<b>Step 2 - Grab the Timeline Modules and Targets</b></h3>
<div>
The Targets in KAPE define what files will be collected. The Modules define what executable will be ran against the files that are collected. </div>
<div>
<br /></div>
<div>
To grab the latest Targets and Module from github, run gkape.exe and click the "Synch with GitHub" at the very bottom of KAPE. This will get you the latest Targets and Modules. If you have downloaded KAPE after 8/27/2019 you can skip this step as they are included in the distro of KAPE already.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiW5P_T5xooyMmGO-6gGc33InJd8pW7LW2QXqU8TpYJAM014JvG5XU90ZdEdiKJM_i9P8W916Wrm_SJb4rHgNpycDS7nsfwpylEhIValZ7LZE5gWLt5TegIQyUWy-aRt77e-O_rGS6-O3L-/s1600/github-synch.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="864" data-original-width="1256" height="220" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiW5P_T5xooyMmGO-6gGc33InJd8pW7LW2QXqU8TpYJAM014JvG5XU90ZdEdiKJM_i9P8W916Wrm_SJb4rHgNpycDS7nsfwpylEhIValZ7LZE5gWLt5TegIQyUWy-aRt77e-O_rGS6-O3L-/s320/github-synch.png" width="320" /></a></div>
<div>
<br /></div>
<div>
The timeline modules I created should be in the \Modules\timelining subfolder. If you do not have this folder after syncing (Eric was working on implement the syncing of module subfolders at the time of this blog post) you will need to grab the timelining folder directly from github: </div>
<div>
<a href="https://github.com/EricZimmerman/KapeFiles/tree/master/Modules">https://github.com/EricZimmerman/KapeFiles/tree/master/Modules</a></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4dmgDAeTl_vOvUBk16A6Y5FW3eFc4k121QwV1ONvWd0eRYVcPDkc2mVphuMCklLLkGEVxkS-PvQRKCwHOb3c6JP5ChAeaNYzbnG-RpE2fACKPok1g4OdOrOZkX3VvxoO1LajEDnbmf-OD/s1600/timelining_modules.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="648" data-original-width="1152" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4dmgDAeTl_vOvUBk16A6Y5FW3eFc4k121QwV1ONvWd0eRYVcPDkc2mVphuMCklLLkGEVxkS-PvQRKCwHOb3c6JP5ChAeaNYzbnG-RpE2fACKPok1g4OdOrOZkX3VvxoO1LajEDnbmf-OD/s320/timelining_modules.png" width="320" /></a></div>
<h3>
Step 3 - Grab the executables</h3>
<div>
The timelining modules will call specific executable to run against the targets. For example, if we want to parse out the eventlogs, the program EvtxEcmd.exe is called by KAPE to parse the artifact. The executables are placed in the KAPE bin folder:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9sT8tg2kXS4DQkLpE8Nubzz2hFg9i97sq9wpeBZk79Lmg2P56la-DhYw3MeoA5LSTEPmMafvR7PX-ElqUDZ1vN2NIuep0lebuU4gq_3QYZIs_FP6rRhg_ZM_Yp-UNtF0JmoRM2soDsDdi/s1600/KAPE+bin+folder.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="561" data-original-width="648" height="277" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9sT8tg2kXS4DQkLpE8Nubzz2hFg9i97sq9wpeBZk79Lmg2P56la-DhYw3MeoA5LSTEPmMafvR7PX-ElqUDZ1vN2NIuep0lebuU4gq_3QYZIs_FP6rRhg_ZM_Yp-UNtF0JmoRM2soDsDdi/s320/KAPE+bin+folder.PNG" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
</div>
<div>
</div>
<div>
For the timeline modules, here are the executable you will need to download, and the locations where they go under the bin folder:</div>
<div>
<br /></div>
<div>
<div>
<b><u>MFTeCmd</u></b></div>
<div>
Purpose: Parse $MFT file</div>
<div>
Download: <a href="https://ericzimmerman.github.io/">https://ericzimmerman.github.io/</a></div>
<div>
Instructions: should already be in \modules\bin. Tested with version 0.4.4.4</div>
<div>
<br /></div>
<div>
<b><u>EvtxECmd.exe</u></b></div>
<div>
Purpose: Parse out *.evtx event logs</div>
<div>
Download: <a href="https://ericzimmerman.github.io/">https://ericzimmerman.github.io/</a></div>
<div>
Instructions: should already be in \modules\bin\EvtxECmd folder. <b>Make sure you have version 0.5.1.0 or newer as the older version will not work for timelining.</b></div>
<div>
<br /></div>
</div>
<div>
<br /></div>
<div>
<b><u>Regripper v 2.8</u></b></div>
<div>
Purpose: Run regripper plugins against various registry hives</div>
<div>
Download: <a href="https://github.com/keydet89/RegRipper2.8">https://github.com/keydet89/RegRipper2.8</a></div>
<div>
Instructions: Make folder \modules\bin\regripper. Place rip.exe , "p2x5124.dll" and the plugins folder in the regripper folder:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmcECPHMKCOI55y4331-1REsO3U8lFdlxdckX35KL31XwcZlZVokOs4rtWwLSGQ99xejnLNv0h9SCoVHGThGOTKO2Pmf8iqPHuFjKS1DCI_peQQ7eLQpBDtZXx86SJ1VRk6IdQDiNYqthr/s1600/KAPE+reg-ripper.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="307" data-original-width="482" height="203" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmcECPHMKCOI55y4331-1REsO3U8lFdlxdckX35KL31XwcZlZVokOs4rtWwLSGQ99xejnLNv0h9SCoVHGThGOTKO2Pmf8iqPHuFjKS1DCI_peQQ7eLQpBDtZXx86SJ1VRk6IdQDiNYqthr/s320/KAPE+reg-ripper.PNG" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<b><u>Harlan Carvey Timeline Tools</u></b></div>
<div>
Purpose: parse event logs, timeline registry hives, convert timeline formats, etc.</div>
<div>
Download: <a href="https://github.com/keydet89/Tools/tree/master/exe">https://github.com/keydet89/Tools/tree/master/exe</a></div>
<div>
Instructions: Create the folder \modules\bin\tln_tools. Place bodyfile.exe, evtparse.exe, parse.exe, regtime.exe and p2x5124.dll in the \tln_tools folder.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPF7OAt2Nebo_2D8uPu2Qu01sgw5175k7yAuckUyLHPpxRvvV5lzvhUL0NMxXGnnyUKCthg3ii-tIHgIE5S2Oa6yTXqDWf-Oczcc2-ljfR5vfjA69Ps4Mq191C9iZvgUpmfLevq7Tb6R5u/s1600/tln_tools.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="263" data-original-width="397" height="211" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPF7OAt2Nebo_2D8uPu2Qu01sgw5175k7yAuckUyLHPpxRvvV5lzvhUL0NMxXGnnyUKCthg3ii-tIHgIE5S2Oa6yTXqDWf-Oczcc2-ljfR5vfjA69Ps4Mq191C9iZvgUpmfLevq7Tb6R5u/s320/tln_tools.png" width="320" /></a></div>
<div>
<b><u>Mari DeGrazia Timeline Tools</u></b></div>
<div>
Purpose: Convert between file formats</div>
<div>
Download: <a href="https://github.com/mdegrazia/KAPE_Tools/tree/master/executables">https://github.com/mdegrazia/KAPE_Tools/tree/master/executables</a></div>
<div>
Instructions: Place in the tln_tools folder created previously</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDWTs7CKULfo0zom0CR4GPqGYnpsr3MrCb1EpsYRYUG3oIS5Jt5QNWA1X7wZ5L6RxNXNDto1ebAmKcx1UZnqcRhuqWzTxY13IknKlpscVtkpl4LTYn2hzn9DYeg5rpDFzVR9k5_YYTK3RI/s1600/tln_tools_degrazia.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="263" data-original-width="397" height="211" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhDWTs7CKULfo0zom0CR4GPqGYnpsr3MrCb1EpsYRYUG3oIS5Jt5QNWA1X7wZ5L6RxNXNDto1ebAmKcx1UZnqcRhuqWzTxY13IknKlpscVtkpl4LTYn2hzn9DYeg5rpDFzVR9k5_YYTK3RI/s320/tln_tools_degrazia.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<b>Step 4 Generate that timeline!</b></div>
<div>
Once you have the the targets,modules and executables set up, you can generate a timeline. </div>
<div>
First, run the Target options in KAPE to grab the triage data. This can be done on a mounted image, or an external USB drive attached to the system of interest. You could even use something like F-Response to run KAPE to do a remote collection.</div>
<div>
<br /></div>
<div>
<b>Check "Use Target Options"</b></div>
<div>
<b>Target Source: </b>The drive letter you want to collect the files from. On a mounted image, this would be the drive letter that the image is mounted as. For a live collection, this would most likely be C:</div>
<div>
<b>Target Destination</b>: This would be where you want the files copied to. Most likely an external drive for a live collection, or a folder on your analysis computer for a mounted image.</div>
<div>
<b>Targets: </b>Select MiniTimelineCollection</div>
<div>
Hit execute when ready:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6cWFYoTx0d-RLcBw0prDYbTyHTHdBX1QE3D7ptGtlZ0bPb6-7EMPFJBftP2d7anmRjWkjOviUaN81KvFvDjSytMKGdTrISQJoVUWNpEBQQpbsXNPFsbJxW_u1i2o0Haoyvr4f9Y0ogYyb/s1600/KAPE-target-collection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="870" data-original-width="1255" height="221" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6cWFYoTx0d-RLcBw0prDYbTyHTHdBX1QE3D7ptGtlZ0bPb6-7EMPFJBftP2d7anmRjWkjOviUaN81KvFvDjSytMKGdTrISQJoVUWNpEBQQpbsXNPFsbJxW_u1i2o0Haoyvr4f9Y0ogYyb/s320/KAPE-target-collection.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
KAPE will now copy all the required files and place them into the Target Destination folder.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3BoH2IfBEAsDWeygGkAS8V7yL3e-od4cnGDCnKo2F2nzyO_EPRjv1UILNQ3QcRLb_pkcq42QecfJ7atA4jV5qwFB1HWWG7-d2XHcZc3X3XrrDjb0gM4UXPfev5tgBt6UP22iMhVHonuC6/s1600/collected_files.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="418" data-original-width="479" height="279" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3BoH2IfBEAsDWeygGkAS8V7yL3e-od4cnGDCnKo2F2nzyO_EPRjv1UILNQ3QcRLb_pkcq42QecfJ7atA4jV5qwFB1HWWG7-d2XHcZc3X3XrrDjb0gM4UXPfev5tgBt6UP22iMhVHonuC6/s320/collected_files.PNG" width="320" /></a></div>
<div>
<br /></div>
<div>
Now that the targets are collected, the modules will need to be run against the collected files. </div>
<div>
<br /></div>
<div>
<b>Select "Use Module Options"</b></div>
<div>
<b>Module Source:</b> This will be where the files were copied to by KAPE </div>
<div>
<b>Module Destination:</b> The will be where the resulting timeline will be created</div>
<div>
<b>Modules:</b> Select Mini_Timeline and if desired, Mini_Timeline_Slice_by_Daterange. The daterange will give you a smaller timeline with a specified date range.</div>
<div>
<b>Variables:</b> Add two variable by using the Key and Value fields. computerName and dateRange. computerName will be the name of the system you are analyzing. The dateRange will be the dateRange you want the smaller timeline to have. This has to be in the format mm/dd/yyyy-mm/dd/yyyy.</div>
<div>
Once completed, select execute:</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgX8_lmvyYR5FBcw2Fh-_Tks8c0z88iQfZ_KqKuzF6XXmpzSDy_34OMDYD6IdqPQRmI_pArnD0pY0VJRpzMoCuwIV19bpwG6fO0xlEI92OP9ZxVZKaJLMTm-XGkPbPqOwFyzPDEfk-JNYMt/s1600/KAPE_modules.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="869" data-original-width="1256" height="221" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgX8_lmvyYR5FBcw2Fh-_Tks8c0z88iQfZ_KqKuzF6XXmpzSDy_34OMDYD6IdqPQRmI_pArnD0pY0VJRpzMoCuwIV19bpwG6fO0xlEI92OP9ZxVZKaJLMTm-XGkPbPqOwFyzPDEfk-JNYMt/s320/KAPE_modules.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
After running, you will have two timelines created in CSV format. The other files created are temporary working files created during the process. These CSV files can be opened using a text editor, TimelineExplorer, Excel, or any other CSV tool of your choice. If you want some more details on how to do the timeline analysis, and where to get started with analysis of these files watch my webex towards the end. </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHO4rLoq2YD1uHi9zueVTfCjalJWahAVgDh8Jbnqtik_kCdabaemIieS1dDACN7DKvPaShWd-k6bXHZkiqq4gru8yvo4vd5uW6nZv3LE2CHKhMmlsq62Tjd7GAlimTA2IAoLHVSMx-UiDx/s1600/timelines.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="295" data-original-width="500" height="188" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHO4rLoq2YD1uHi9zueVTfCjalJWahAVgDh8Jbnqtik_kCdabaemIieS1dDACN7DKvPaShWd-k6bXHZkiqq4gru8yvo4vd5uW6nZv3LE2CHKhMmlsq62Tjd7GAlimTA2IAoLHVSMx-UiDx/s320/timelines.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
I highly recommend reading Harlan Carvey's <a href="http://windowsir.blogspot.com/2009/02/timeline-analysis.html">blog posts on timelineing,</a> as KAPE is just a way to automate this process.</div>
<div>
<br /></div>
<div>
Here is a detailed breakdown of what the timeline targets and modules I created are doing:<br />
<br />
<b>Targets collect the following files:</b><br />
$MFT<br />
Registy hives (SAM,SECURITY, SOFTWARE, SYSTEM, NTUSER.DAT, UsrClass.dat)<br />
Event logs (*.evt, *.evtx)<br />
<br />
<b>Timeline Modules will include the following in the timeline:</b><br />
File MACB timestamps<br />
Last write times of the above registry hive's keys<br />
ReRipper plugins ran: muicache, userassist, AppCompatCache, Services<br />
Event Logs with Event ID and descriptions<br />
<br />
<br />
For more detailed information on KAPE, including how to write modules and targets, check out the KAPE <a href="https://ericzimmerman.github.io/KapeDocs/">documentation</a>.</div>
<div>
<br /></div>
Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com0tag:blogger.com,1999:blog-2456910508235577340.post-32010993405813742852018-06-13T06:49:00.001-07:002018-06-13T07:09:57.232-07:00Malicious PowerShell in the Registry: Persistence<div class="separator" style="clear: both; text-align: center;">
</div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoqX3q8KyrhPL5Eo-nNQP6GHAFLUL21TFkF6eOxqfOw_z8GQNwoSCjYPsDVtgsVPsMPzr-FHHEbhsQBzC7I2w2VdFGCRmTzSzwmp0krITEOs_FQTeCjewJvF0ku8QNYgtbsTPc3ydIpOQ1/s1600/powershell_registry.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="739" data-original-width="1001" height="147" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhoqX3q8KyrhPL5Eo-nNQP6GHAFLUL21TFkF6eOxqfOw_z8GQNwoSCjYPsDVtgsVPsMPzr-FHHEbhsQBzC7I2w2VdFGCRmTzSzwmp0krITEOs_FQTeCjewJvF0ku8QNYgtbsTPc3ydIpOQ1/s200/powershell_registry.png" width="200" /></a>This is the second part in my series on Finding and Decoding Malicious PowerShell Scripts. My <a href="http://az4n6.blogspot.com/2017/10/finding-and-decoding-malicious.html">first blog</a> post walked through how to find malicious PowerShell scripts in the System event log, and the various steps to decode them. In this post, I wanted to discuss another location where malicious PowerShell scripts might be hiding - the Registry.<br />
<br />
The Registry is a great place for an attacker to establish persistence. Popular locations for this are the Run keys located in either the Software Hive, or in a User's ntuser.dat hive. For a list of run keys, check out the <a href="https://www.forensicswiki.org/wiki/Windows_Registry#Run_keys">Forensic Wiki.</a><br />
<br />
A technique I've seen in some cases I've worked is an attacker using PowerShell in the Run key to call <i><b>another</b></i> key that contains the base64 code that contains a payload. <br />
<br />
Let's see what an example of this looks like. Using Eric Zimmerman's <a href="https://ericzimmerman.github.io/">Registry Explorer</a> I've navigated to the following registry key: HKLM\Software\Microsoft\Windows\CurrentVersion\Run. Underneath the value "hztGpoWa" the following entry is made:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkvWw7JSP5VET_IDqUK3J4WpxFFsNYbYjBlo7EVOqPPmz3Gw_GIY5KcPfNrLUhRXm-jrEj5ARdChqzYbV4p-_3gsX66qhwC1arSjSzm63uVjphftVGt97WE2vSt1FHh7vo9Lk1WAc6NYf-/s1600/powershell_in_registry_1.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="404" data-original-width="1170" height="220" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkvWw7JSP5VET_IDqUK3J4WpxFFsNYbYjBlo7EVOqPPmz3Gw_GIY5KcPfNrLUhRXm-jrEj5ARdChqzYbV4p-_3gsX66qhwC1arSjSzm63uVjphftVGt97WE2vSt1FHh7vo9Lk1WAc6NYf-/s640/powershell_in_registry_1.PNG" width="640" /></a></div>
<br />
You can also use Harlan's <a href="https://github.com/keydet89/RegRipper2.8">RegRipper's</a> soft_run plugin to pull this information:<br />
<br />
<div class="code">
rip.exe -r SOFTWARE -p soft_run</div>
<br />
Output:<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjEZnyhyphenhyphenD1o91vSsF3lQVjzKmTdrzZ1UIBZQK452AQabK2VHg3B5N517Voq4mNtf18PwSPR1ZLVya5x_yNFl6YMyHB33LALHd18XzxQEZQz-vUcEH7wQ0mZIlXRr9i5MBX4FuSEeGViQHa/s1600/rr_output.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="174" data-original-width="864" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjEZnyhyphenhyphenD1o91vSsF3lQVjzKmTdrzZ1UIBZQK452AQabK2VHg3B5N517Voq4mNtf18PwSPR1ZLVya5x_yNFl6YMyHB33LALHd18XzxQEZQz-vUcEH7wQ0mZIlXRr9i5MBX4FuSEeGViQHa/s1600/rr_output.PNG" /></a><br />
(for the NTUSER.DAT hive, use the user_run plugin)<br />
<br />
So what does this command do? %COMSPEC% is the system variable for cmd.exe. This uses cmd.exe to launch PowerShell in a hidden window. It then uses the PowerShell command "Get-Item" to get <i><b>another</b></i> registry key<b> - </b>HKLM:Software\4MX64uqR, and the value Dp8m09KD under that key. <br />
<br />
Browsing to the HKLM:Software\4MX64uqR key in Registry Explorer reveals a whole mess of base64:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_zqWD74k_uyv2iOUHeH4vD5WaKzmw3__UAb8kRqVrL-soVRvnoBKvNYzsPBFIFCDEXcgD1w954rMKD2Kb164Pn6stI3nxLZUx5bElWYRRAjMP343jGEktJDnF9VDiqxpjYciwu6j3ne1h/s1600/b64_registry.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="691" data-original-width="997" height="442" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_zqWD74k_uyv2iOUHeH4vD5WaKzmw3__UAb8kRqVrL-soVRvnoBKvNYzsPBFIFCDEXcgD1w954rMKD2Kb164Pn6stI3nxLZUx5bElWYRRAjMP343jGEktJDnF9VDiqxpjYciwu6j3ne1h/s640/b64_registry.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
Another way to pull base64 like this from the registry is to use the "sizes" plugin from <a href="https://github.com/keydet89/RegRipper2.8">RegRipper</a>. This will search the registry hive for values over a certain threshold and dump them out:<br />
<br />
<div class="code">
rip.exe -r SOFTWARE -p sizes</div>
<br />
(A thanks to Harlan for updating this plugin! Make sure to update it if you haven't recently.)<br />
<br />
To see the detailed steps of how to decode this base64, take a look at my <a href="http://az4n6.blogspot.com/2017/10/finding-and-decoding-malicious.html">earlier blog post</a> on decoding malicious PowerShell scripts.<br />
<br />
Here are the high-level steps to decode it:<br />
<ul>
<li>Decode unicode base64 in registry key</li>
<li>Decode and decompress (gzip) embedded base64 </li>
<li>Decode another round of embedded base64</li>
<li>payload = shellcode</li>
<li>Try running scdb.exe or strings over shellcode for resulting IP address and port</li>
</ul>
The resulting code more often than not is a way to establish a Meterpreter reverse shell.<br />
<br />
Another way to find instances of malicious PowerShell in the registry is to search the registry for "%COMSPEC%".<br />
<br />
I used <a href="https://ericzimmerman.github.io/">Registry Explorer</a> and it's handy Find command to do this. Make sure and have the right "Search in" boxes selected:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUPjq7Hj1tnjMQD-1Nh00RFDoAZkrx-Das1CUl9l0KdFCpdX2767_sZXyyB5ieJz6qoJGg1anmDQcutJ6O_uhoaV3XtxS4XBGYjUhiTjYXIq-DBsXgRMp3zRvXKvFQwQCpH0j65nWHj21N/s1600/comspec_in_registry.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="667" data-original-width="906" height="470" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhUPjq7Hj1tnjMQD-1Nh00RFDoAZkrx-Das1CUl9l0KdFCpdX2767_sZXyyB5ieJz6qoJGg1anmDQcutJ6O_uhoaV3XtxS4XBGYjUhiTjYXIq-DBsXgRMp3zRvXKvFQwQCpH0j65nWHj21N/s640/comspec_in_registry.PNG" width="640" /></a></div>
<br />
While this example showed registry keys and values with random names - this is not always the case. These names can be whatever the attacker wants and they will not always be an obvious tip off like a random name.<br />
<br />
For my example, I used Metasploit to install this persistence mechanism in the registry. Check out all the options available. As mentioned above, the registry key/value names may be set to anything:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRc9CsIxzi0ct3p0VpnpnwsumDmC1_G1blDPqp0wXVkflLOy4uxhLI9WyjU1rD-Y99Aw0yGZbM48_8XO47oENcubDEIFcRXqT4n06CdPVqsF4EKSBxJKEVFJLSVHi3xp2KBF12wMCOX2q-/s1600/metasploit.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="187" data-original-width="1102" height="108" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRc9CsIxzi0ct3p0VpnpnwsumDmC1_G1blDPqp0wXVkflLOy4uxhLI9WyjU1rD-Y99Aw0yGZbM48_8XO47oENcubDEIFcRXqT4n06CdPVqsF4EKSBxJKEVFJLSVHi3xp2KBF12wMCOX2q-/s640/metasploit.PNG" width="640" /></a></div>
<br />
My next post on malicious PowerShell scripts will cover PowerShell logging and pulling information from memory. Happy Hunting!<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com0tag:blogger.com,1999:blog-2456910508235577340.post-79489200796411933332018-01-05T07:50:00.000-07:002018-02-01T15:01:02.054-07:00Mounting an APFS image in LinuxAs a follow up to my post on how to <a href="http://az4n6.blogspot.com/2018/01/how-to-mount-mac-apfs-images-in-windows.html">mount AFPS images</a> on Windows, I wanted to post about how to mount an APFS image on a Linux system. If you are looking for how to mount an APFS image on a Mac, Sarah Edwards wrote a awesome <a href="https://www.mac4n6.com/blog/2017/11/26/mount-all-the-things-mounting-apfs-and-4k-disk-images-on-macos-1013">blog post</a> on how to do this. There is also another one over at <a href="https://www.blackbagtech.com/blog/2017/10/05/apple-file-system-apfs-mac-forensic-imaging-analysis/">BlackBag.</a>If you are new to APFS, I would also recommend an informative <a href="https://www.youtube.com/watch?v=OX5H-RsKexI">video</a> by Steve Whalen where he explains APFS in detail. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGqnVGCh_FRs7YRFD2XllVdGv4EcP1Sm1i42JDl-HaW22CpCUbLYMzVDJ5q7UO_eC_ALYVxGTtI6YUkrd7CspwNBIbs3B2w1yRnAa80SmAEQATaBN9Kni4VDHxfvxntntAdoJQtYSvuiTU/s1600/linux_mac.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="244" data-original-width="207" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGqnVGCh_FRs7YRFD2XllVdGv4EcP1Sm1i42JDl-HaW22CpCUbLYMzVDJ5q7UO_eC_ALYVxGTtI6YUkrd7CspwNBIbs3B2w1yRnAa80SmAEQATaBN9Kni4VDHxfvxntntAdoJQtYSvuiTU/s1600/linux_mac.jpg" /></a></div>
Options, options, options. It's always nice to have options in forensics. Sometimes one way may not work for you, or maybe you don't have access to a Mac at the moment. If you are on a Windows machine and need access to an APFS volume or image (E01 or raw), it's easy enough to spin up a Linux VM and get to work.<br />
<br />
For my testing, I used an experimental Linux APFS driver by sgan81 - <a href="https://github.com/sgan81/apfs-fuse">apfs-fuse</a>. Note the word <b>"experimental"</b> - and read the disclaimers by the author. I would strongly recommend verifying any results with another tool or method, such as the one detailed by Sarah Edwards. However, this method works in a pinch, and at least you can start analysis until you get things working on a Mac. Oh - and according to the documentation, it will prompt you for a password if the volume is encrypted.<br />
<br />
These instructions assume that you already have an image of the Mac, either in E01 or raw format (dd, dmg, etc). For my Linux distro, I used the free <a href="https://digital-forensics.sans.org/community/downloads">SIFT Workstation</a> Virtual Machine on Ubutnu 16.04. If you are using another Linux distro, you may need to install additional dependencies, etc.<br />
<br />
<h3>
Preparing the SIFT Workstation</h3>
First things first, some dependencies need to be installed before apfs-fuse will work. As always, run sudo apt-get update before installing any dependencies:<br />
<br />
<div class="code">
sudo apt-get update<br />
sudo apt-get install libattr1-dev
</div>
<br />
If you are running a version of SIFT prior to the one based on Ubuntu 16.04, a couple of additional dependencies may be needed. This includes a newer version of cmake. This can be installed by following the instructions on the <a href="https://cmake.org/install/">cmake</a> website. In addition to cmake, older version of SIFT may also need the the ICU library:<br />
<br />
<div class="code">
sudo apt-get install libicu-dev</div>
<span style="color: red;"><br /></span>
<br />
<h3>
Download and build apfs-fuse</h3>
Next, download the apfs-fuse driver from github:<br />
<br />
<div class="code">
git clone https://github.com/sgan81/apfs-fuse<br />
<br /></div>
Now compile it, and install it:<br />
<br />
<div class="code">
cd apf-fuse<br />
mkdir build<br />
cd build<br />
cmake ..<br />
make</div>
<pre></pre>
<pre></pre>
<h3>
Mounting the E01 Image</h3>
Now that the SIFT workstation has been set up, we can mount the E01 image. If you have a dd/raw image, you can skip to the next step.<br />
<br />
I like using the ewfmount tool in SIFT to mount E01s. Once mounted, there will be a "virtual" raw image of the E01 file under the designated mount point. The syntax is simple, and works on split images as well (just specify the first segment for split images).<br />
<br />
syntax:<br />
ewfmount <image name> <mount point><br />
example:<br />
<div class="code">
ewfmount mac_image.E01 /mnt/ewf</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3KzrDPsYXCMu2gRY1zt772XOFI9IlefOgjJW6mA2D9zBJifmI0viN46OMbLl9JiQ-U40WqY7ICzRbP4S_pdAbSqz7eQA5efVoa3ItMlrZBwuGqqVs5ZLachpgD8OZyYbRHXuK5e_4bUn5/s1600/ewfmount.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="236" data-original-width="828" height="182" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3KzrDPsYXCMu2gRY1zt772XOFI9IlefOgjJW6mA2D9zBJifmI0viN46OMbLl9JiQ-U40WqY7ICzRbP4S_pdAbSqz7eQA5efVoa3ItMlrZBwuGqqVs5ZLachpgD8OZyYbRHXuK5e_4bUn5/s640/ewfmount.PNG" width="640" /></a></div>
<br />
If you have issues with ewfmount, check out this <a href="https://digital-forensics.sans.org/blog/2011/11/28/digital-forensic-sifting-mounting-ewf-or-e01-evidence-image-files">blog post</a> for some alternative tools to mount ewf files.<br />
<br />
<h3>
Mounting the raw image to a loopback device</h3>
Now that we have a dd/raw image to work with - either from mounting the E01, or because that is how the image was taken - we'll mount it to a loopback device. The Linux apfs-fuse driver needs the volume where the APFS container is. Because the disk image may contain additional partitions, we will need to figure out the offset where the APFS partition begins.<br />
<br />
Below is a screen shot in X-Ways. Here was can see that X-Ways identified an APFS partition starting at sector 76,806 as well as 4096 bytes per sector (note, although X-Ways identified the partition as being APFS, it did not parse it out). <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgC40nLNxjYTUBH6cgZSvC8tj2UEbeKWqp5jMUW0XMToS7Z4hkBxPoPgKFpw1HoAzwqzQ6iu2GUifXlPfPU6WyNBKkcWqeXzS2BV1sKwLtABw8TDra1yke0VEilBpniDP-9hyphenhyphenyPafOMqUnO/s1600/x-ways.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="874" data-original-width="1319" height="424" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgC40nLNxjYTUBH6cgZSvC8tj2UEbeKWqp5jMUW0XMToS7Z4hkBxPoPgKFpw1HoAzwqzQ6iu2GUifXlPfPU6WyNBKkcWqeXzS2BV1sKwLtABw8TDra1yke0VEilBpniDP-9hyphenhyphenyPafOMqUnO/s640/x-ways.PNG" width="640" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
Alternatively, we can use the Sleuthkit tool mmls to list the partitions on the image. Here was can see that there is a "NoName" partition that starts right after the EFI System Partition. The offset is 76806 and is the largest partition on the drive. The Units are also displayed as 4096 bytes per sector:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRK2oI-32TleD9fUtr4wZPQeb1O43LIKoN5IcCYDRKOd4VMrkJdTMd8_4yiUpTeum43qGaSOk8L8pcXqzIjBBE7ICxXi3N-MtCpLPy1qibTk2x6wYeTy6dfbmq4FnBChmcjJLsWQlAPj8M/s1600/mmls_apfs_image.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="298" data-original-width="828" height="230" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRK2oI-32TleD9fUtr4wZPQeb1O43LIKoN5IcCYDRKOd4VMrkJdTMd8_4yiUpTeum43qGaSOk8L8pcXqzIjBBE7ICxXi3N-MtCpLPy1qibTk2x6wYeTy6dfbmq4FnBChmcjJLsWQlAPj8M/s640/mmls_apfs_image.PNG" width="640" /></a></div>
<br />
To run mmls on the mounted EWF:<br />
<div class="code">
mmls /mnt/ewf/ewf1</div>
To run mmls on a dd/raw image:<br />
<div class="code">
mmls mac_image.dd</div>
<br />
<br />
To set up the looback device, we will need to supply the APFS starting partition offset in bytes. Since the offset is given in sectors, we will need to convert from sectors to bytes by multiplying 4096 bytes/per sector times the number sectors:<br />
4096 X 76806 = 314597376<br />
<br />
Armed with this information, we can mount the "NoName" partition, aka the APFS partition, to a loopback device:<br />
<br />
For the mounted EWF file: <br />
<div class="code">
losetup -r -o 314597376 /dev/loop0 /mnt/ewf/ewf1</div>
for the dd/raw image:<br />
<div class="code">
losetup -r -o 314597376 /dev/loop0 mac_image.dd </div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgzz_rpOug4P1EOppeeItLFpY-et2QCk0Rgf4OWsdSURLpJA9OXXRnuusrZlrPCc0NtsX07DDNiOtfN13yk7ZmKecmTPfI7NuUWjpFVHC3lJ1GqhJwzzSf6JOL0EQEMRFye8oel9sz2OBr/s1600/apfs_loopback_mount.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="112" data-original-width="828" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgzz_rpOug4P1EOppeeItLFpY-et2QCk0Rgf4OWsdSURLpJA9OXXRnuusrZlrPCc0NtsX07DDNiOtfN13yk7ZmKecmTPfI7NuUWjpFVHC3lJ1GqhJwzzSf6JOL0EQEMRFye8oel9sz2OBr/s1600/apfs_loopback_mount.PNG" /></a></div>
<br />
In the syntax above, -r is read only, and -o is the offset in bytes to the start of the APFS partition.<br />
<br />
<b>Mount up the APFS filesystem</b><br />
<br />
Ok! Finally! Now we are ready to mount up the APFS partition to the filesystem. The apf-fuse binary will be in a folder name "bin" within the build folder created earlier when the apfs-driver was installed. Change into that directory, and run apfs-fuse by pointing it to the loopback device and a mount point:<br />
<br />
<div class="code">
mkdir /mnt/apfs<br />
./apfs-fuse /dev/loop0 /mnt/apfs</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWgqfQLPsgc63Y_wT1Q6ta2amZFmEA15GS8TqrOFEwNbur1eazX3XCxSY9O91lTjqJq0iD4m_ZUtwuhVWIyQYVdoKw_Bk1JMkbG3wOdEOFKkl4r5DdiMZzt9SFMLtI4lDGiDINHqV8kC-b/s1600/apfs-fuse.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="106" data-original-width="824" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWgqfQLPsgc63Y_wT1Q6ta2amZFmEA15GS8TqrOFEwNbur1eazX3XCxSY9O91lTjqJq0iD4m_ZUtwuhVWIyQYVdoKw_Bk1JMkbG3wOdEOFKkl4r5DdiMZzt9SFMLtI4lDGiDINHqV8kC-b/s1600/apfs-fuse.PNG" /></a></div>
<br />
<br />
In my testing, the cursor just blinks and does not give a status message. I opened another terminal and did an ls command on the mount point to see if it mounted ok:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCzu9TDboEtCn0ZFtVCFmSaU3_5xJuTaGrmfxvlr5np68i2MI5eRLXKhTbNc9lSct4ymU3GkphZ-c7TVwnr62GuV9Fovf6UqjIQ9O9PePwAL7Q6wS8CvxcVKrFhZJW2WdVaiywPXaq97P6/s1600/apfs-fuse-mount-list.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="209" data-original-width="829" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCzu9TDboEtCn0ZFtVCFmSaU3_5xJuTaGrmfxvlr5np68i2MI5eRLXKhTbNc9lSct4ymU3GkphZ-c7TVwnr62GuV9Fovf6UqjIQ9O9PePwAL7Q6wS8CvxcVKrFhZJW2WdVaiywPXaq97P6/s1600/apfs-fuse-mount-list.PNG" /></a></div>
<br />
Success! Now I can run AV Scans, view files, and export out any files as needed.<br />
<br />
As I mentioned before - this is an experimental driver and all results should be verified. Hopefully as time passes we will have more ways to mount and access APFS images in Linux, and our mainstream tools.<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com2tag:blogger.com,1999:blog-2456910508235577340.post-66630641002780865632018-01-01T13:32:00.002-07:002018-01-01T13:32:41.034-07:00How to mount Mac APFS images in WindowsAPFS is the new file system for Mac OS, and so far, many forensic suites are playing catch up as far as support goes. As such, workarounds may need to be employed in order to conduct analysis on Mac OS APFS images. This short blog post will cover one of those workarounds - mounting an APFS image in Windows.<br />
<br />
Paragon has a free (preview) driver to mount APFS volumes in Windows!!!! Sweet!!!<br />
<br />
APFS for Windows is going to look for a connected APFS drive. Since we have an image, we will need to mount the image as a SCSI device so the Windows APFS driver can see it. To do this, we will use <a href="https://arsenalrecon.com/weapons/image-mounter/">Arsenal Image Mounter.</a><br />
<br />
<br />
Mount the image using Arsenal Image Mounter. I had to select the sector size of 4096 for it to work since the sector size in my image was 4096 (If you need to know the sector size of your image, you can use a tool like <a href="http://www.sleuthkit.org/sleuthkit/man/mmls.html">mmls </a>to check).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEge3UePjZ2XdgFwMzeVV1C1Le8rOCjCVO4vgAsHC5hziREQyrwCNUyeFPfwPsZpP7sZkBmqtyWQYflOJYHnVjwW1G0FlJ2Bw5SdRoUqisVpXV6tFK89mBgsLsQBdSym0x-GeHpd8MoK5k93/s1600/aresenl+image+mounter+4096.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="536" data-original-width="882" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEge3UePjZ2XdgFwMzeVV1C1Le8rOCjCVO4vgAsHC5hziREQyrwCNUyeFPfwPsZpP7sZkBmqtyWQYflOJYHnVjwW1G0FlJ2Bw5SdRoUqisVpXV6tFK89mBgsLsQBdSym0x-GeHpd8MoK5k93/s640/aresenl+image+mounter+4096.PNG" width="640" /></a></div>
<br />
<br />
Download and install <a href="https://backstage.paragon-software.com/home/apfs-windows/">APFS for Windows from Paragon</a> and launch it. It should automatically detect the APFS volume:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGYwfTY9TrjKT43ErJbnyun20wjHiRFiPswQD76z4ilqiIfFZXReS1YlYxMK-KsEgcfjJ0bCNYibkAY7Mh3C7fqrKOtSoxakbXo9JUT-FpyYkB9Be8KVCp6zh1PYBsa1DNavG0tuOFhIm1/s1600/paragon_apfs_driver.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="592" data-original-width="995" height="380" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGYwfTY9TrjKT43ErJbnyun20wjHiRFiPswQD76z4ilqiIfFZXReS1YlYxMK-KsEgcfjJ0bCNYibkAY7Mh3C7fqrKOtSoxakbXo9JUT-FpyYkB9Be8KVCp6zh1PYBsa1DNavG0tuOFhIm1/s640/paragon_apfs_driver.PNG" width="640" /></a></div>
<br />
Now you can browse the APFS drive in Windows:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmNlv2j3QZrRHd8dBEhL_khNjakPHY5hurO0spdYOFTXER9XZ0WUgYHSbPMl4CQPy7lwlnMtp3aMSy_L3-1_Mtj5L3_YkMxGPzYEA7fpAdiXuDMGUq93wO4652xLloi54ZUCjVXB5-LD3K/s1600/apfs.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="455" data-original-width="1028" height="282" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmNlv2j3QZrRHd8dBEhL_khNjakPHY5hurO0spdYOFTXER9XZ0WUgYHSbPMl4CQPy7lwlnMtp3aMSy_L3-1_Mtj5L3_YkMxGPzYEA7fpAdiXuDMGUq93wO4652xLloi54ZUCjVXB5-LD3K/s640/apfs.PNG" width="640" /></a></div>
<br />
<br />
And add it to your favorite all in one tool, like X-Ways, as a logical drive:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0eHWf-o7ySjje1f0RnK2hGzLYjExvDjUtA7fImhGG3NltRL8ayMLb0bLF5wfIMXtilnvfFZsvDaq8ktcauwpvo0TpebeL7BHwOauGm6k00uUdaviwNsOQ4K5Hqg3IiIGPb3zfXtjCE97g/s1600/apfs_windows_mount.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="453" data-original-width="682" height="424" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0eHWf-o7ySjje1f0RnK2hGzLYjExvDjUtA7fImhGG3NltRL8ayMLb0bLF5wfIMXtilnvfFZsvDaq8ktcauwpvo0TpebeL7BHwOauGm6k00uUdaviwNsOQ4K5Hqg3IiIGPb3zfXtjCE97g/s640/apfs_windows_mount.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<br />
Happy Hunting!Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com8tag:blogger.com,1999:blog-2456910508235577340.post-87625849834749473072017-10-16T06:54:00.001-07:002017-10-16T07:03:18.130-07:00Finding and Decoding Malicious PowerShell Scripts<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
PowerShell. It's everywhere. I've started coming across more and more malicious PowerShell scripts. <br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Why do attackers love using PowerShell? Because it's native to many versions of Windows, provides full access to the WMI and .Net Framework and can execute malicious code in memory thereby evading AV. Oh yeah - did I mention a lack of logging too?<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqeTwANNzUnPj6M41edzw3pQ7Jd7Qt9SXmdx7xBiWh8t9mrSmEOaG9WGaX2uvC_v9UOKsmX8gx92iUEbsnxTwS6hAHw4Q4B3TAygXyyDAo6qRCRwHQO-t0onrfkUmX_ChvLmTSQPPRhmdm/s1600/powershell-skull.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="501" data-original-width="675" height="147" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqeTwANNzUnPj6M41edzw3pQ7Jd7Qt9SXmdx7xBiWh8t9mrSmEOaG9WGaX2uvC_v9UOKsmX8gx92iUEbsnxTwS6hAHw4Q4B3TAygXyyDAo6qRCRwHQO-t0onrfkUmX_ChvLmTSQPPRhmdm/s200/powershell-skull.png" width="200" /></a>During the course of my analysis on these types of cases, I have found several indications that PowerShell has been utilized by an attacker. These include installed services, registry entries and PowerShell scripts on disk. If logging is enabled, that can provide some nice artifacts as well. The perspective of my post is going to be from that of an analyst that may not be too familiar with PowerShell. I am going to discuss how I locate malicious PowerShell artifacts during my analysis, as well as some methods I use to decode obfuscated PowerShell scripts. This will be Part 1 in a 3 part series written over the next few weeks.<br />
<br />
<b><u>Part 1: PowerShell Scripts Installed as Services </u></b><br />
First up to bat is my favorite - PowerShell scripts that I find as installed services in the System event log. To find these, one of the first things I do is look for Event ID 7045. This event occurs when a service is installed on a system. An example of a PowerShell script installed as a service is shown below:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPv5-83Uo7dBA6N7pL9PF2FwEy9ImSXJOQiyLqL-yY5dvUEu_ZHvp2dzThzdc8w0O60l06VE7HlGTdOFf-A7_JdJZA-pRf-Jw79bE0p1QCj6GUklHcefNT1FTTCY5TnxzqZjjXZMvu5ixr/s1600/base64_encoded_powershell.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="821" data-original-width="971" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPv5-83Uo7dBA6N7pL9PF2FwEy9ImSXJOQiyLqL-yY5dvUEu_ZHvp2dzThzdc8w0O60l06VE7HlGTdOFf-A7_JdJZA-pRf-Jw79bE0p1QCj6GUklHcefNT1FTTCY5TnxzqZjjXZMvu5ixr/s1600/base64_encoded_powershell.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
Of note are the following red flags:<br />
<br />
1) Random Service Name<br />
2) The Service File Name has "%COMSPEC%", which is the environment variable for cmd.exe<br />
3) A reference to the powershell executable<br />
4) Base 64 encoded data <br />
<br />
So how might an entry like this make its way into an event log? While there are various ways to do this, one method would be to use the built in Windows Service Control Manger to create a service:<br />
<br />
<div class="code">
sc.exe create MyService binPath=%COMSPEC% powershell.exe -nop -w hidden -encodedcommand <insertbase64><br />
<br />
sc start MyService</div>
<br />
The above commands create a service named "MyService" and uses the binPath= option to launch cmd.exe which in turns executes the PowerShell code. <br />
<br />
An interesting thing to note - there may be some failed errors logged after the service is created in this manner. The errors <i><b>do not mean</b></i> that it was unsuccessful. Windows was just expecting a "real" service binary to be installed and "times out" waiting for the "service" to report back. How do I know this? In my testing I was able to set up a successful reverse shell using the above methodology, which generated a failed service error on the Windows machine. On the left is a Metaspolit session I started on an attack virtual machine. On the right is a Windows 7 host virtual machine. Although the Windows 7 machine states "The service did not respond to the start or control request in a timely fashion," a reverse shell was still opened in the Metatsploit session:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqXpx2d8Mp9SucZRckyMmTtHqxBZjn2Tvrby-SuzgG6n-DCt1dhrnfCxBXEiUUe4ZOQt-MncJY58TT9t4jjyDF1jny7RpMTDsz9z9Z1kDP9HXQls0J3esrDApB0r5Nvp8zm7aYS0QgQuAg/s1600/virtual_machines.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="850" data-original-width="1600" height="338" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqXpx2d8Mp9SucZRckyMmTtHqxBZjn2Tvrby-SuzgG6n-DCt1dhrnfCxBXEiUUe4ZOQt-MncJY58TT9t4jjyDF1jny7RpMTDsz9z9Z1kDP9HXQls0J3esrDApB0r5Nvp8zm7aYS0QgQuAg/s640/virtual_machines.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Below are the two corresponding event log entries, 7000 and 7009, made in the System event log. Although the 7009 message states "The FakeDriver service failed to start.." this does not mean that the command inside the binPath variable did not execute successfully. So beware, interpreting these as in indication that the PowerShell did not execute<span style="color: red;"> <span style="color: black;">may</span></span> be false: <br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipuVuZTPeLrLphwti0p8UyieMFwfNHco0l5mk13AKksvTdHv591TK9qY5iGut6f6shsas6Z1CBEzb_pkayZcYZPAv6ulYzVcKIbBXh2IM6RCtgjTlNsQ1sTgQ9zXT96e6YvUpJJxfr1qoZ/s1600/7009Error.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="362" data-original-width="608" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipuVuZTPeLrLphwti0p8UyieMFwfNHco0l5mk13AKksvTdHv591TK9qY5iGut6f6shsas6Z1CBEzb_pkayZcYZPAv6ulYzVcKIbBXh2IM6RCtgjTlNsQ1sTgQ9zXT96e6YvUpJJxfr1qoZ/s1600/7009Error.PNG" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGx0yl3U9Uvnf3CRvPDCEbBlJcaEXvscZ5GHEfBPO0xP6PzcAZe5AdFoNof6wbtaJCIjV_2fO9a6hvA381o01Zwl4D1jVMLyM1S3uqneVJjJN6JyTXHPxQfKquWb4lBd94HptMNDVbz0rL/s1600/7000_error.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="342" data-original-width="601" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGx0yl3U9Uvnf3CRvPDCEbBlJcaEXvscZ5GHEfBPO0xP6PzcAZe5AdFoNof6wbtaJCIjV_2fO9a6hvA381o01Zwl4D1jVMLyM1S3uqneVJjJN6JyTXHPxQfKquWb4lBd94HptMNDVbz0rL/s1600/7000_error.PNG" /></a></div>
<br />
<br />
The 7045 System event log PoweShell command is encoded in base64 and python can be used to decode it. Interesting note - this base64 code is in Unicode, so there will be extra parameter specified when decoding it. (For display reasons I have truncated the base64 text - you would need to include the full base64 text to decode it): <br />
<br />
<div class="code">
import base64<br />
code="JABjACAAPQAgAEAAIgAKAFsARABsAGwASQBtAHAA...."<br />
base64.b64decode(code).decode('UTF16')</div>
<br />
Here is what the decoded PowerShell command looks like. A quick sweep of the code reveals some telling signs - references to creating a Net Socket with the TCP protocol and an IP address: <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcz40uR5GONKoFDFVcxNg-zDtANUEEHyruM2UuTIGsAitiWjwrB18db5pfhJBJ2KmEghQAg0Ke0gEdPoQ7aSHcWF_BE6JGV3iHGBkyetBHb0DfV1amLjfj0K5ijRtmaWXevJeH7IBfSoCg/s1600/decoded_powershell_b64_notes.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="301" data-original-width="801" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcz40uR5GONKoFDFVcxNg-zDtANUEEHyruM2UuTIGsAitiWjwrB18db5pfhJBJ2KmEghQAg0Ke0gEdPoQ7aSHcWF_BE6JGV3iHGBkyetBHb0DfV1amLjfj0K5ijRtmaWXevJeH7IBfSoCg/s1600/decoded_powershell_b64_notes.PNG" /></a></div>
<br />
This is similar to the type of code that Meterpreter uses to set up a reverse shell. The above PowerShell code was pretty easy to decode, however, it's usually more involved.<br />
<br />
Next up is another example - this time its just "regular" base64. Note again the %COMSPEC% variable and reference to powershell.exe: <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjN2e9aARmcwEGiYBuu0fk12aaFkno7qivDSBBKT2cBoHyEquytiy3Iu9buO7ytsYdnpMYjCuxGOrh2o0LF1rvZkgY-h6VRmN-vO-Fv-ojshNMIKuXPRLd_Unz6N1iYX1LoJUSPcfXfwtGR/s1600/base64_encoded_example2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="443" data-original-width="957" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjN2e9aARmcwEGiYBuu0fk12aaFkno7qivDSBBKT2cBoHyEquytiy3Iu9buO7ytsYdnpMYjCuxGOrh2o0LF1rvZkgY-h6VRmN-vO-Fv-ojshNMIKuXPRLd_Unz6N1iYX1LoJUSPcfXfwtGR/s1600/base64_encoded_example2.PNG" /></a></div>
<br />
Again, Python can be used to decode the base64 encoded PowerShell:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2lsqljR6La8THGx-GqTTA3yI7XDSU8tLsZp3cfB7ml6z2if9P7dPxUOhpPfClGfV3yu8Pgo_9WqI2Fy9-xYV0DCJCdBT-VMTSnG50FcIWRgRE0QruSLv5kiIeWf0qnzAaqy2j-dAoWFjD/s1600/base64_decoded_2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="264" data-original-width="807" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2lsqljR6La8THGx-GqTTA3yI7XDSU8tLsZp3cfB7ml6z2if9P7dPxUOhpPfClGfV3yu8Pgo_9WqI2Fy9-xYV0DCJCdBT-VMTSnG50FcIWRgRE0QruSLv5kiIeWf0qnzAaqy2j-dAoWFjD/s640/base64_decoded_2.PNG" width="640" /></a></div>
<br />
This time, the decoded output is less than helpful. If we go back and take a look at the System event log entry more closely, we can see that there are references to "Gzip" and "Decompress":<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgACWktwhHgnJcOVXC6ECaQn0JlRj7KuQjVQ3IWYqDK0eQCHT-lFfkdGZLvsTISK1u5zUcz8THCJHg0b1C812jYSg_TdkxiM6fdxQgsCZmFREP2gi6_czDEicCQ3JCjV-nMvev_GejLKAS/s1600/base64_encoded_example2_b.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="443" data-original-width="957" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgACWktwhHgnJcOVXC6ECaQn0JlRj7KuQjVQ3IWYqDK0eQCHT-lFfkdGZLvsTISK1u5zUcz8THCJHg0b1C812jYSg_TdkxiM6fdxQgsCZmFREP2gi6_czDEicCQ3JCjV-nMvev_GejLKAS/s1600/base64_encoded_example2_b.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Ahh.. so thinking in reverse, this data may have been compressed with Gzip then encoded using base64. Using python, I am going to write out the decoded base64 into a file so I can try unzipping it:<br />
<br />
<div class="code">
import base64<br />
code="H4sICCSPh1kCADEAT..."<br />
decoded=base64.b64decode(code)<br />
f=open("decoded.gzip",'wb')<br />
f.write(decoded)<br />
f.close</div>
<br />
Using 7zip I am successfully able to unpack the gzip file! Since I did not get any errors, I may be on the right track:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDSPKmueYQjS5oLZZIi2Q2bZuk08r9lkCxJmHrwOqV7DwFw5cJ0GYNEQmSOvc_mFmeQCPXKvL3iGN9pdwz8sQlcFByLXYNHW7QRK1XpjNV_V3KNCn8olRGTfHXNZgY_62Wy60JpaCw6dNG/s1600/zip1.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="451" data-original-width="753" height="382" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDSPKmueYQjS5oLZZIi2Q2bZuk08r9lkCxJmHrwOqV7DwFw5cJ0GYNEQmSOvc_mFmeQCPXKvL3iGN9pdwz8sQlcFByLXYNHW7QRK1XpjNV_V3KNCn8olRGTfHXNZgY_62Wy60JpaCw6dNG/s640/zip1.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
Now if I open the unzipped file with a text editor, hopefully I will see some PowerShell code:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuBKe1aPHRt6-AeA547H0fQOw0cwyQ0EvqlphqHAKSA5kSZiSNj3ADPDAH4VQ6VO5RcN73dATGhCsnWjOtnCJFb4t3o7SbIj3n5msDc9OVrdMXEKU-LJIk9PaovhCNMz082TLGspj9OfJu/s1600/1.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="382" data-original-width="732" height="332" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuBKe1aPHRt6-AeA547H0fQOw0cwyQ0EvqlphqHAKSA5kSZiSNj3ADPDAH4VQ6VO5RcN73dATGhCsnWjOtnCJFb4t3o7SbIj3n5msDc9OVrdMXEKU-LJIk9PaovhCNMz082TLGspj9OfJu/s640/1.PNG" width="640" /></a></div>
<br />
<br />
Ahh..what??? Ok - time to take a peek in a hex editor:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjliZDzbPnEHjZfEy3QJdcgUvYNWawDIc7e0EBrarCDDVK5vcZiFr6JE-Uv5SulMfJZAFeRYwsYXiTwltWU0_fF2AiV1T-sTwoIIBMSBgDg4fPWL_5cg1AYuSDygAY9z0p0Qbk65XqdPGM6/s1600/1_in_hex.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="513" data-original-width="676" height="484" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjliZDzbPnEHjZfEy3QJdcgUvYNWawDIc7e0EBrarCDDVK5vcZiFr6JE-Uv5SulMfJZAFeRYwsYXiTwltWU0_fF2AiV1T-sTwoIIBMSBgDg4fPWL_5cg1AYuSDygAY9z0p0Qbk65XqdPGM6/s640/1_in_hex.PNG" width="640" /></a></div>
<br />
Not much help either. I am thinking this may be shellcode. As a next step, I am going to run it through <a href="http://sandsprite.com/blogs/index.php?uid=7&pid=57">PDF Stream Dumper's</a> shellcode analysis tool, scdbg.exe:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWxHaofubbDmPruFLU4USPbWDL7CKEutv0vXbcZQAO_a5pqQfe5JbhCpwUU0t00h6vvKWyXs0eRJIZ6nTW6_a_zj6xgZj_bi6hYOGo0M54_kDrrYv_eQUSxLqnteO3LHcuDLEe8pOGOWNj/s1600/shellcode.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="463" data-original-width="1030" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWxHaofubbDmPruFLU4USPbWDL7CKEutv0vXbcZQAO_a5pqQfe5JbhCpwUU0t00h6vvKWyXs0eRJIZ6nTW6_a_zj6xgZj_bi6hYOGo0M54_kDrrYv_eQUSxLqnteO3LHcuDLEe8pOGOWNj/s1600/shellcode.png" /></a></div>
<br />
<br />
Ta-Da! scdbg.exe was able to pull out some IOCs for me from the shellcode. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
To summarize, here are the steps I took took to decode this PowerShell entry:<br />
<ul>
<li>Decoded the base64 PowerShell string</li>
<li>Wrote out the decoded base64 to a zip file</li>
<li>Decompressed the Gzip file using 7zip</li>
<li>Ran the binary output through scdbg.exe</li>
</ul>
As demonstrated above, there can be several layers to get though before the golds strikes. <br />
<br />
One final example:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZkKRMU5fwvgz61MOlmHNyRjqKPQr0O-d2RSz8IKZImiJCAnMbvm0gyEP-DRm6CrqF-EdmyOCIJJPhhL9e6SXhvElM3iM-3G28pW81s4io7hZf5StGmKvkvb9W7vE6HH3soZOjUkoBRKwu/s1600/base64_3.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="728" data-original-width="956" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZkKRMU5fwvgz61MOlmHNyRjqKPQr0O-d2RSz8IKZImiJCAnMbvm0gyEP-DRm6CrqF-EdmyOCIJJPhhL9e6SXhvElM3iM-3G28pW81s4io7hZf5StGmKvkvb9W7vE6HH3soZOjUkoBRKwu/s1600/base64_3.PNG" /></a></div>
<br />
This looks familiar. First step, decoding the Unicode base64 gives the following result - which contains <b>more base64 code inside the base64 code</b>! :<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4uIOeE0wm8qZp4Wl06RefD5FjK9FtTf38bfUbwinZRPFXtQFyAT2bnM39UeSqDxSUpDlewvGrT4c-wkcTWbwZI7FydN1jrjay6gHwu-xewAl4W4GXTDUvvK8tBnZsadl_aEEeY5U77lBT/s1600/example_3_b64_decoded.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="233" data-original-width="1176" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4uIOeE0wm8qZp4Wl06RefD5FjK9FtTf38bfUbwinZRPFXtQFyAT2bnM39UeSqDxSUpDlewvGrT4c-wkcTWbwZI7FydN1jrjay6gHwu-xewAl4W4GXTDUvvK8tBnZsadl_aEEeY5U77lBT/s1600/example_3_b64_decoded.PNG" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Obfuscated, then obfuscated again with compression. This is very typical to what I have seen in cases. This time because there is no reference to "gzip" in the compression text, I am just going to save the second round of base64 to a regular zip file and try to open again with 7zip:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="code">
decoded2="nVPvT9swEP2ev+IUR...."<br />
f=open("decoded2.zip,"wb")<br />
f.write(base64.b64decode(decoded2)<br />
f.close()<br />
<br /></div>
When trying to open up the zipped file with 7Zip I get an error:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8b7OJcfswc6hKeiOiWcGMKqblg042kFVD6MGtSxZxdyTuUh6FoKJPUoNxt164g43gEYOecO9YgZJfbcSE5d98JAaqe7_iEnGCTkPwDMkzxfBg-HBSd4q6LkYuiNBdgWN7twuN2DJgBGNp/s1600/zip_error.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="362" data-original-width="671" height="344" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8b7OJcfswc6hKeiOiWcGMKqblg042kFVD6MGtSxZxdyTuUh6FoKJPUoNxt164g43gEYOecO9YgZJfbcSE5d98JAaqe7_iEnGCTkPwDMkzxfBg-HBSd4q6LkYuiNBdgWN7twuN2DJgBGNp/s640/zip_error.PNG" width="640" /></a></div>
<br />
And the same with the built in Window's utility:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeTtcCn3Re5R7ZYruT6d5I8FbBThZn60mCczfHUdrsz2sJ5zFSLJ4b4zl0YbP8wlBWszkb-3PGERRhVZM_vq3ty5NDrPsAKNcTK3QcdwYZwj-miyTwNJHIdkK5pmxYbs5pwUSMF4veXGXA/s1600/zip_error_windows.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="454" data-original-width="614" height="472" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeTtcCn3Re5R7ZYruT6d5I8FbBThZn60mCczfHUdrsz2sJ5zFSLJ4b4zl0YbP8wlBWszkb-3PGERRhVZM_vq3ty5NDrPsAKNcTK3QcdwYZwj-miyTwNJHIdkK5pmxYbs5pwUSMF4veXGXA/s640/zip_error_windows.PNG" width="640" /></a></div>
<br />
I also tried various python libraries to unzip the compressed file. After some research, I discovered that the compression used is related to some .Net libraries. Now, since I am a python gal, I wanted to figure out how to decompress this using Python so I could easily implement it into my scripting. Since Python is cross compatible with Linux, Windows and Mac, .Net is not native to its core. As such, I used <a href="http://ironpython.net/">Iron Python</a> to do my bidding. (Now yes, you could absolutely use PowerShell to decode this, but what can I say - I wanted to do it Python)<br />
<br />
According to the <a href="http://ironpython.net/">Iron Python website</a> "IronPython is an open-source implementation of the Python programming language which is tightly integrated with the .NET Framework. IronPython can use the .NET Framework and Python libraries, and other .NET languages can use Python code just as easily." Neat. Installing it on Windows is a breeze - just an MSI. Once installed, you simple run the scripts calling ipy.exe (I'll show an example later).<br />
<br />
Armed with this, I was able to write some python code (io_decompress.py) that decompressed the zip file using the python IO compression Library:<br />
<br />
<div class="code">
#import required .Net libraries <br />
<br />
from System.IO import BinaryReader, StreamReader, MemoryStream<br />
from System.IO.Compression import CompressionMode, DeflateStream<br />
from System import Array, Byte<br />
from System.IO import FileStream, FileMode<br />
from System.Text import Encoding<br />
from System.IO import File<br />
<br />
#functions to decompress the data <br />
def decompress(data):<br />
io_zip = DeflateStream(MemoryStream(data), CompressionMode.Decompress)<br />
str = StreamReader(io_zip).ReadToEnd()<br />
io_zip.Close()<br />
return str<br />
<br />
print "Decompressing stream..."<br />
compressedBytes = File.ReadAllBytes("decoded2.zip")<br />
decompressedString = decompress(compressedBytes)<br />
<br />
f = open("decompressed.txt", "wb")<br />
f.write(decompressedString)<br />
f.close()</div>
<br />
To run the script using IronPython was easy: ipy.exe io_decompress.py:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQRvgv40b_181rHiChGE2WiRAZeINnTxADsjbfxiZqOCFaiszgl4q8G7nkzwcYcp65CFnOA_R07OqTfHeAMrIZAbqPYb-9vIQfjiGs_XDx4zzFfVzTqu1pRCNEJinzF8MNR99jd6hgfqcT/s1600/iron_python.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="177" data-original-width="820" height="138" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQRvgv40b_181rHiChGE2WiRAZeINnTxADsjbfxiZqOCFaiszgl4q8G7nkzwcYcp65CFnOA_R07OqTfHeAMrIZAbqPYb-9vIQfjiGs_XDx4zzFfVzTqu1pRCNEJinzF8MNR99jd6hgfqcT/s640/iron_python.PNG" width="640" /></a></div>
<br />
I was able to open the decompressed.txt file created by the script and was rewarded with the following plain text PowerShell script. Once again, note the IP address:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHYDybSXm7U1Tuk8iQ3LPiKf0YQFcV_NAryDByjJkODCIsh8EPy7ZRp4vK9wWKBmV1gVQpbcqP5oJbcsXzU6LeoTzK5J_f3zU4B96FWriOyOmW4L8wy_lnXl92QSKbTLZbP6ucf-wZxClN/s1600/decompressed_text.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="405" data-original-width="963" height="267" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHYDybSXm7U1Tuk8iQ3LPiKf0YQFcV_NAryDByjJkODCIsh8EPy7ZRp4vK9wWKBmV1gVQpbcqP5oJbcsXzU6LeoTzK5J_f3zU4B96FWriOyOmW4L8wy_lnXl92QSKbTLZbP6ucf-wZxClN/s640/decompressed_text.png" width="640" /></a></div>
<br />
To summarize the steps taken for this event log entry:<br />
<ul>
<li>Decoded Unicode base64</li>
<li>Decoded embedded base64 code</li>
<li>Decompressed resulting decoded base64 code</li>
</ul>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcjyKGvBsoRaBQcc3DQVZwsP3WIWVz_Rh6FxQ3Us2ncIIS8AV3athHTfRHO4YDXcY0bZSxyNExpxOVJIUhs1CmGNybJbSOxJdXMVjmLvVpQRquIINUv0QhSOParUHBuKOambdnU-sqNlzI/s1600/russion_stacking_dolls.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="516" data-original-width="801" height="128" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcjyKGvBsoRaBQcc3DQVZwsP3WIWVz_Rh6FxQ3Us2ncIIS8AV3athHTfRHO4YDXcY0bZSxyNExpxOVJIUhs1CmGNybJbSOxJdXMVjmLvVpQRquIINUv0QhSOParUHBuKOambdnU-sqNlzI/s200/russion_stacking_dolls.png" width="200" /></a>As we have seen from the three examples above, there are various techniques attackers may use to obfuscate their PowerShell entries. These may be used in various combinations, some of which I have demonstrated above. The steps taken vary for each case, and within each case itself. I usually see 2-3 variations in each case that are pushed out to hundreds of systems over the course of several months. Sometimes the steps might be:base64, base64,decompress, shellcode. It might also be: base64, decompress, base64, code, base64, shellcode. See how quickly this becomes like a Matryoshka doll? When I wrap up the series, I will talk about ways to automate the process. If you are using something like Harlan Carvy's <a href="http://windowsir.blogspot.com/p/timelines.html">timelines scripts</a> to get text outputs, it becomes pretty easy. <br />
<br />
So how to go about finding these and decoding them in your exams? <br />
<br />
<ul>
<li>Look for event log ID 7045 with "%COMSPEC%, powershell.exe, -encodedcommand, -w hidden , "From Base64String" etc.<br /> </li>
<li>Look for "Gzipstream" or "[IO.Compression.CompressionMode]::Decompress" for hints on what type of compression was used</li>
<li>Try running the resulting binary files through sdbg.exe, shellcode2exe or other malware analysis tools</li>
</ul>
<br />
Part 2 will be about PowerShell in the registry, followed by Part 3 on PowerShell logging and pulling information from memory.<br />
<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com4tag:blogger.com,1999:blog-2456910508235577340.post-1409954674522754932017-02-24T08:20:00.000-07:002017-02-24T08:20:50.321-07:00Onion Peeler: Batch Tor Lookup ProgramLogs, Logs, Logs. I see, IPs. When reviewing log files for suspect activity it can be helpful to look up information related to IP addresses. There is a great utility for this by Nirsoft called <a href="http://www.nirsoft.net/utils/ipnetinfo.html">IPNetinfo</a>. You can import a whole list of IP addresses and it will give you "the owner of the IP address, the country/state name, IP addresses range, contact information
(address, phone, fax, and email), and more."<br />
<br />
When I am reviewing log files, an IP address associated with a foreign country may peak my interest. Another check I like to do is look for activity associated with Tor nodes. In a corporate environment, a user accessing a system from a Tor exit node may be a red flag.<br />
<br />
When I am checking an IP address to see if it is associated with a Tor exit node I will use a website like<a href="https://exonerator.torproject.org/"> ExoneraTor</a>. It lets me put in an IP address and a date, and lets me know if the IP address is associate with a Tor relay. While this is a great tool, if I have a list of IP addresses to check, it's not very efficient. To that end, I wrote a little program to help automate the process of checking a list of IP addresses against Tor Relays and Bridges, Onion Peeler.<br />
<br />
Onion Peeler is written in Python and uses <a href="https://github.com/duk3luk3/onion-py">OnionPy</a>. OnionPy is a wrapper for the <a href="https://onionoo.torproject.org/">OnionOO</a> Tor Api. Using OnionPy, Onion Peeler caches a local copy of the Tor exit nodes and performs a check for a list of supplied IP addresses. What's nice is that if you have a list of sensitive IPs, the information is not shared and is kept locally:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivgDpMar2Sc42o-pEIj2WOUa1PFs4OviOKWxHsE2lyfd65VOrqqamoG5oGS4ppmLTKzPQqtq1P141rZjRLICt8rDPAdKIXGyLhYDeCpFSXRItFSHgeShShsnuFbFlqXWPe3qBFny7RPE7q/s1600/onion_peeler.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivgDpMar2Sc42o-pEIj2WOUa1PFs4OviOKWxHsE2lyfd65VOrqqamoG5oGS4ppmLTKzPQqtq1P141rZjRLICt8rDPAdKIXGyLhYDeCpFSXRItFSHgeShShsnuFbFlqXWPe3qBFny7RPE7q/s1600/onion_peeler.png" /></a></div>
<br />
It will output a list of matches:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgX_SJhaZzIVHdKSKzURtl8ybl1rz9mFqbWBwKzxjEu-VMcG0k7cwpIq7YZhgDP11T0VMkcSSUd_PikoNZRNxLGiFK5U5uCeVl6EoW36sWdZ1HUAB2-EN6EkSkIaac3jAlkKZve9G99oc_5/s1600/output.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="128" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgX_SJhaZzIVHdKSKzURtl8ybl1rz9mFqbWBwKzxjEu-VMcG0k7cwpIq7YZhgDP11T0VMkcSSUd_PikoNZRNxLGiFK5U5uCeVl6EoW36sWdZ1HUAB2-EN6EkSkIaac3jAlkKZve9G99oc_5/s640/output.png" width="640" /></a></div>
<br />
<br />
<br />
Since it's in Python, the program is cross-platform compatible. I've tested it on Windows, Linux and Mac. It just requires OnionPy, which can be installed using "pip install OnionPy". I also have a compiled Windows Executable if you don't have Python installed. It requires an Internet connection as the initial query grabs the latest Tor nodes from OnionOO. I am thinking about adding in a way to store an offline copy in the next version as well as add in additional details about the Tor nodes (first seen, last seen etc.)<br />
<br />
It took about a minute to check 8,000 IP addresses. Of course, a bigger list will take longer, so be patient. <br />
<br />
Code and program are available on <a href="https://github.com/mdegrazia/OnionPeeler">my github.</a>Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com2tag:blogger.com,1999:blog-2456910508235577340.post-53357543568296741272017-02-20T14:43:00.000-07:002017-02-20T14:48:25.809-07:00When Windows LiesWait, What? Windows lies? I believe so...<br />
<br />
I worked a case where I checked the Windows Install date and it was a couple days before we received the
system. GREAT....did the user reformat their drive and do a fresh install before handing over the laptop? Did they reinstall the OS? This would not have been the first time a laptop or system was rebuilt after an incident (either on purpose or by accident).<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7xK7-CFqRkS928rz8jlyJ4qKh_DgrmE510qIgIsW4c65U1TWuVVE3o6FK59hSRsaRoAHxCgYzDGZO-SUnHnSRAteu0_jQXNP6utrIfqnjPPAsV9VokWBhTF-CcjGywDZPwmo4Xf7evC_g/s1600/nose.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7xK7-CFqRkS928rz8jlyJ4qKh_DgrmE510qIgIsW4c65U1TWuVVE3o6FK59hSRsaRoAHxCgYzDGZO-SUnHnSRAteu0_jQXNP6utrIfqnjPPAsV9VokWBhTF-CcjGywDZPwmo4Xf7evC_g/s1600/nose.png" /></a></div>
<br />
Checking basic information like the Operating System and Installation date can help a examiner prioritize the systems they need to examine and check for evidence spoliation issues. If you have 20 systems to go through, and the Operating System has been installed AFTER the date of the incident you may want to focus on some other systems first. In civil or criminal cases, an installation date right before you receive the evidence may raise some red flags. <br />
<br />
So now that we have established why the Operating Installation date can be important, there is a registry key you can retrieve it from, HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion. RegRipper has a great plugin for this, named OSVersion that pulls not only the OS version, but the install date from the registry key . Running this against my test system I got the following output:<br />
<br />
----------------------------------------<br />
winver v.20081210<br />
(Software) Get Windows version<br />
<br />
ProductName = Windows 10 Home<br />
<span style="background-color: yellow;">InstallDate = Fri Feb 3 15:58:47 2017</span><br />
---------------------------------------- <br />
<br />
<span style="color: red;"><span style="color: black;">What??? Install date of February 3rd, 2017?? 2 weeks ago??? Since this is my system, I know I did not install Windows 10 February 3rd. I have had Windows 10 Home since the roll out in 2015!</span></span><br />
<br />
<span style="color: red;"><span style="color: black;">Just to be a thorough, I verified the date was being parsed correctly and looked at the raw data in the registry:</span></span><br />
<span style="color: red;"><span style="color: black;"> </span> </span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEharKHHJ8M8odyJxNRIkDr-3ZVOrb0NqOu280rh0xcqNyswAd_RW3p4bBKllzH6AMeL4H3qzFO-h1mzHW3fVfRKXDVWaKTCK5oT5aclKLBfMWihHkneluylwRlw_-AomWvXyQMevwAMkzv7/s1600/Operating_Install_Date.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="428" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEharKHHJ8M8odyJxNRIkDr-3ZVOrb0NqOu280rh0xcqNyswAd_RW3p4bBKllzH6AMeL4H3qzFO-h1mzHW3fVfRKXDVWaKTCK5oT5aclKLBfMWihHkneluylwRlw_-AomWvXyQMevwAMkzv7/s640/Operating_Install_Date.PNG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
Install date is 0x5894a8b7 which is Fri, 03 February 2017 15:58:47 UTC.<br />
<br />
Ok, one last check... running the systeminfo command it clearly shows "Original Install Date" as 2/3/2017 8:58:47 AM. I am UTC -7, so this information matches my above results.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8xAEVqeK7fhL9TMeClBmIAuKOMTeFchVZzVXZHFFhM30WjkpNUVeOqTZhVzSdKArK_zngsc3kF2O17F-9Uo4-j65Zli3C4c5BabBmSx-_kD9a8AYB-BM8G2Wmzv8_qOZcSzY0GNJWCanO/s1600/system_info_red.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj8xAEVqeK7fhL9TMeClBmIAuKOMTeFchVZzVXZHFFhM30WjkpNUVeOqTZhVzSdKArK_zngsc3kF2O17F-9Uo4-j65Zli3C4c5BabBmSx-_kD9a8AYB-BM8G2Wmzv8_qOZcSzY0GNJWCanO/s1600/system_info_red.PNG" /></a></div>
<br />
<br />
OK - Windows, I think you are lying to me. I'm hurt. But, to add insult to injury, with some more digging around I find out that YOU MESSED WITH MY LOGS during this supposed install. <br />
<br />
I have a snapshot of my system previous to the supposed install date of February 3rd, 2017. Note the created dates on the Event Logs - 12/12/2015 and a size of 20MB:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpn4eHn3XCHfIMC9jlyh96Ay3bM3krkjmC1ipSxFlfDETIqTG_-gdstTglr1dAinnlfmqqDHGEwyS10asax8k5bjnMkJbB99VGgFDMqCIQ3BcPYaw4lrbzbQEobFpPOk8A-2jNDoiyV7EC/s1600/event_logs_before_red.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="414" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpn4eHn3XCHfIMC9jlyh96Ay3bM3krkjmC1ipSxFlfDETIqTG_-gdstTglr1dAinnlfmqqDHGEwyS10asax8k5bjnMkJbB99VGgFDMqCIQ3BcPYaw4lrbzbQEobFpPOk8A-2jNDoiyV7EC/s640/event_logs_before_red.png" width="640" /></a></div>
<br />
Now I look at the current created dates and files sizes of my event logs. Note that the created date is the same of this supposed install date, 2/3/2017. Not only that, but my log files are much smaller, some about 2MB:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxkhc2isk57g6T0010jLRRm1KQ9CmhTkDbV0jJ4fCKlj4gO6PwdeNyxOzTh8OAY3y5h7g4bgN788uh6sWAjVwY4xY5yFmA6bqrpzpETbltO74gOkAsGBLUXAHHWMuTJ-mSikRTz1uh9sD9/s1600/event_logs_after_red.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="380" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxkhc2isk57g6T0010jLRRm1KQ9CmhTkDbV0jJ4fCKlj4gO6PwdeNyxOzTh8OAY3y5h7g4bgN788uh6sWAjVwY4xY5yFmA6bqrpzpETbltO74gOkAsGBLUXAHHWMuTJ-mSikRTz1uh9sD9/s640/event_logs_after_red.PNG" width="640" /></a></div>
<br />
When I opened my logs, as expected, there are no entries before 2/3/2017, and the first entry matches with this supposed install date:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFK_chXVSNnwoQEMBEHxkmLbEJxDrBWW4vHaBXuMrYotd1Q1aqubP-5wIEFbnVFPmdiygogzJhBgZx_MMtmMGmpfHYrxBEfcESWx_X4W6eIfmnjBwNanFndtx74r_LCuoYjdQ8KIJRgCuY/s1600/security.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="274" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFK_chXVSNnwoQEMBEHxkmLbEJxDrBWW4vHaBXuMrYotd1Q1aqubP-5wIEFbnVFPmdiygogzJhBgZx_MMtmMGmpfHYrxBEfcESWx_X4W6eIfmnjBwNanFndtx74r_LCuoYjdQ8KIJRgCuY/s640/security.PNG" width="640" /></a></div>
<br />
I was curious what may have caused this. Since Windows updates have caused issues with artifact timestamps before, such as USB devices, I checked the Windows Update history. Sure enough, there was a Windows update, "Feature update to Windows 10, version 1607" which ran on 2/3/2017.This date matches the supposed install date: <br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg__HvKtMgsVju7qs6QipMm64uysEN3O-aYXMJFIogfyX5jJj8t4qSC0L5upKAFW_wxSUISG2Mrty-jCys6yY9DrJVOrqylgspSzC5-jDhHKX0rSq2BxUznM7oytQjrmJHIV_BWqE4Tm9zR/s1600/update_history.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="398" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg__HvKtMgsVju7qs6QipMm64uysEN3O-aYXMJFIogfyX5jJj8t4qSC0L5upKAFW_wxSUISG2Mrty-jCys6yY9DrJVOrqylgspSzC5-jDhHKX0rSq2BxUznM7oytQjrmJHIV_BWqE4Tm9zR/s640/update_history.PNG" width="640" /></a></div>
<br />
Since my update history contained more than one update ran on 2/3/17, I wanted to check some other Windows 10 systems to see what I could find out. I knew approximately when both of these systems had their operating systems installed, and both had incorrect installation dates listed, as well as the Feature Update v. 1607 that ran on the same day. <br />
<br />
<br />
<u><b>System 1 (Windows 10 Home)</b></u><br />
Registry Install Date: 9/27/2016 11:22:39<br />
Event Log created Dates: 9/27/2016 11:11<br />
Feature update to Windows 10, version 1607: 9/27/2016<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi95QYyWCz-ly4Krz3ETShIVCidXMm-5u4CSHxOq0VH56i45NUpcbz-Th8h8imQHxrjMJ5kabRR8G7Qx2a1M_pLIQa8p-ez6kcA0X7g2UVJa5fgFHjhI-8fW4ONV7j7b0sjmMSCYsOOAnIs/s1600/updaet_history.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi95QYyWCz-ly4Krz3ETShIVCidXMm-5u4CSHxOq0VH56i45NUpcbz-Th8h8imQHxrjMJ5kabRR8G7Qx2a1M_pLIQa8p-ez6kcA0X7g2UVJa5fgFHjhI-8fW4ONV7j7b0sjmMSCYsOOAnIs/s640/updaet_history.PNG" width="474" /></a></div>
<br />
<br />
<u><b>System 2 (Window 10 Pro)</b></u><br />
Registry Install Date: 10/1/2016 3:47<br />
Log created Dates: 10/1/2016 3:42<br />
Feature update to Windows 10, version 1607: 10/1/2016<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-rQFQgho2Kn1uNyb9a7u_iLSqeTOQzsdg3LcvLipJiSCGhhKuYk1J55eJUeu5iSe7jrOs5h_BU9KzHpgheUO-Be9j5MiNVzbnp5_yS2zKm24xzjDgHF4SMnly6czPAPUzQWmISZ973ELm/s1600/update_history.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="498" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-rQFQgho2Kn1uNyb9a7u_iLSqeTOQzsdg3LcvLipJiSCGhhKuYk1J55eJUeu5iSe7jrOs5h_BU9KzHpgheUO-Be9j5MiNVzbnp5_yS2zKm24xzjDgHF4SMnly6czPAPUzQWmISZ973ELm/s640/update_history.PNG" width="640" /></a></div>
<br />
<br />
So, my working hypothesis is that the <a href="https://technet.microsoft.com/en-us/itpro/windows/whats-new/whats-new-windows-10-version-1607">Feature update to Windows 10, version 1607</a> is updating the Windows Installation time and deleting the logs.<br />
<br />
This may just be a matter of semantics.. maybe "Operating Install Date" really means - latest major version update??? This artifact may be open to misinterpretation if that is the case. <br />
<br />
Why is this important?<br />
<br />
<b>Possible Incorrect conclusion of evidence spoliation </b><br />
Imagine you are working a civil case where the opposing side is supposed to produce and turn over a laptop. If you see the installation date was recent, you might incorrectly conclude that they installed the OS right before handing over the system. Or, in Incident Response you may incorrectly assume that the operating system was just installed and there may not be too many goodies to find.<br />
<br />
<b><br /></b>
<b>You loose event logs</b><br />
Event logs can make up a critical component of an exam. In the case I was working, this update happened right before I received the laptop. The event logs only had about a day in them. Yes, there may be backups, or a SIEM collecting them, but it just makes the exam more involved. <b> </b><br />
<br />
<b>Other Artifacts????</b><br />
These are just the two inconsistencies I have found so far... there are probably more...<br />
<b> </b><br />
I would like to test this more formally by setting up a virtual machine and tracking the updates to see what happens, however, based upon the systems I have looked at, I think Windows is lying.<br />
<br />
As always, correlating findings against multiple artifacts could help determine if this install date is accurate. <br />
<br />
Just a note and something else to be aware of - in many corporate environments the Operating System install date may be incorrect due to clones/images being used to push out machines. However, I don't consider this as Windows lying because the date would reflect the install date of the original before it was cloned.<br />
<br />
<br />
<b></b>Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com10tag:blogger.com,1999:blog-2456910508235577340.post-55758835483511721952016-10-05T08:22:00.002-07:002016-10-05T08:22:32.246-07:00Quicklook thumbnails.data parser<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbsEy1QdUo7tCjHYwCNeJZr_npQlstqC1zubo03hhWyGeYtscHAAh-FkDMfVkDJHkrYQR86crDniB1yRIip1gJuTyNuyG1whMN0pP3v_espgdgP1iiOF29ImBca0dbAw3GITXnrahSPI5v/s1600/quicklook_thumbnails.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="216" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhbsEy1QdUo7tCjHYwCNeJZr_npQlstqC1zubo03hhWyGeYtscHAAh-FkDMfVkDJHkrYQR86crDniB1yRIip1gJuTyNuyG1whMN0pP3v_espgdgP1iiOF29ImBca0dbAw3GITXnrahSPI5v/s320/quicklook_thumbnails.jpg" width="320" /></a></div>
Earlier this year at the request of a reader I wrote a tool to parse the Quicklook thumbnails index.sqlite file. This sqlite database stores information related to thumbnails that have been generated on a system. This information includes filename, paths, timestamps and more (see my previous <a href="http://2016/05/quicklook-python-parser-all-your-blobs.html">blogpos</a><a href="https://www.blogger.com/null">t</a> for more details). The file is located under
/private/var/folders/<random>/<random>/C/com.apple.QuickLook.thumbnailcache.<br />
<br />
Someone else recently reached out to me and asked about the thumbnails.data file in the same folder, which holds the actual thumbnails. They were having issues carving the images out of this file.<br />
<br />
<b>Research </b><br />
<br />
With a hex viewer, I opened a thumbnails.data file from my Mac and scrolled though the file. I didn't notice any typical image file headers as I scrolled. Below is a screen shot of what I was seeing:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6C_oN_RvHv8nGnbLHkvf-HZlAkdA5XJwG0WjsDpsrP6FpFCvLU045aKMEXJKSjKSAIyjUT03JFaCrjgNY5_YLuwcwWhRbfBLY4rS1DHTvmfvnAsQE84zWLpxTK7kqXwr5OK_kK6mtchUO/s1600/thumbnails_data_hex.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="295" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg6C_oN_RvHv8nGnbLHkvf-HZlAkdA5XJwG0WjsDpsrP6FpFCvLU045aKMEXJKSjKSAIyjUT03JFaCrjgNY5_YLuwcwWhRbfBLY4rS1DHTvmfvnAsQE84zWLpxTK7kqXwr5OK_kK6mtchUO/s400/thumbnails_data_hex.png" width="400" /></a></div>
<br />
<br />
Normally I would expect to see something like the following in hex view:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR0VzTzeJV_LEUMj1VsajzTxPT35izFtK2qONrEvsm96QdapEvzwR9Xv5Rm9c79jmdDpygpcWI6TKZ7hHwTcbsXRfe8D1RLdXxTgJNUz1eO4VgeceRtozI03OjKJSHtoEjDa9TuudGse4q/s1600/png_file_header.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjR0VzTzeJV_LEUMj1VsajzTxPT35izFtK2qONrEvsm96QdapEvzwR9Xv5Rm9c79jmdDpygpcWI6TKZ7hHwTcbsXRfe8D1RLdXxTgJNUz1eO4VgeceRtozI03OjKJSHtoEjDa9TuudGse4q/s1600/png_file_header.PNG" /></a></div>
<br />
Here the file header shown in red is for a PNG file. I tried looking for the file headers for various other images as well, such as jpg, gifs and bmps but no luck.<br />
<br />
I placed the com.apple.QuickLook.thumbnailcache folder on a wiped SD card and used a couple of carving programs to try and carve the images out of the thumbnails.data file. While the carvers were able to carve out the sqlite database, they did not find any images.<br />
<br />
Interesting. I started to do some research, and I found a reference on <a href="http://encase-forensic-blog.guidancesoftware.com/2014/05/examination-of-mac-os-x-quick-look.html">this blog post</a> that the images are stored as "raw bitmaps". "Raw bitmaps" are<i><b> </b></i>not the same as .bmp files. Raw bitmaps do not have a file header or footer and can not be decoded without external information. According to <a href="http://fileformats.archiveteam.org/wiki/Raw_bitmap">this website</a> the following are characteristics of a typical raw bitmap:<br />
<br />
<ul>
<li> No header or footer
</li>
<li> Uncompressed
</li>
<li> Does not use a color palette
</li>
<li> Cannot be decoded without external information, such as:
<ul>
<li> Color type and sample order (RGB, BGR, grayscale, etc.)
</li>
<li> Image width in pixels
</li>
<li> Row padding logic
</li>
<li> <span style="color: black;">Byte order and/or bit order</span></li>
</ul>
</li>
</ul>
This explains why the file carvers I used were not able to carve out the images. File carvers need a file header in order to identify and carve out files. So if we don't have a file header, how do we carve out these images? Luckily, the Quicklook index.sqlite table stores the "external" information needed to carve the images in the thunmbnails.data file. <br />
<br />
This external information includes the bitmap location in the file, the length of the bitmap, width, and height.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwxOsHlg9RTnkKHkhqelRRmi5zxRjtzBGCAUPSOXvorXZ4MUu1KJRK_p9RKD3hN_MkZpAdkNF3Hj1R7l0lrSQX2FQkuysKU029we4COyglMnwOBMj7vZWziaSXywgnDIjuWjZ9poob1SRW/s1600/thumbnails_database.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgwxOsHlg9RTnkKHkhqelRRmi5zxRjtzBGCAUPSOXvorXZ4MUu1KJRK_p9RKD3hN_MkZpAdkNF3Hj1R7l0lrSQX2FQkuysKU029we4COyglMnwOBMj7vZWziaSXywgnDIjuWjZ9poob1SRW/s1600/thumbnails_database.PNG" /></a></div>
<br />
<br />
<b>Manually Carving </b><br />
<br />
Below is an example of how this data looks from the database. For this example, the file file3251255366828.jpg actually has two thumbnails associated with it. One the is 64 X 64, and a larger one that is 164 X 164:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjX2mZTXdjXxXMDTSXCBxPtqeOp02BLSrHC1p7g3mfO5Na3uwYP1dFt6osvKJDpv0rbztxhyphenhyphenXFs2dn0XOJj5xNAg8JoNkqMEWkXHqpN-IxYgJ-nHMiXigh4DQtXc9YD5P04_uiQveSDViRu/s1600/thumbnails_data.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjX2mZTXdjXxXMDTSXCBxPtqeOp02BLSrHC1p7g3mfO5Na3uwYP1dFt6osvKJDpv0rbztxhyphenhyphenXFs2dn0XOJj5xNAg8JoNkqMEWkXHqpN-IxYgJ-nHMiXigh4DQtXc9YD5P04_uiQveSDViRu/s1600/thumbnails_data.PNG" /></a></div>
<br />
I am going to walk through how to manually carve out the 64 X 64 thumbnail using the information from the Quicklook index.sqlite database. While I have written a parser to automate these steps (covered further below), I think it's nice to know how to manually do it so you can validate any tools you may use to do this, or if a tool doesn't work.<br />
<br />
<br />
The first thing is to open up the thumbnails.data file with FTK Imager using the File > Add Evidence Item > Contents of a Folder. To get to the file offset, choose the file, right click in the hex area, then choose<b> Go to offset... </b> The offset we want is the value in the thumbnails table bitmapdata_location field, 993284:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBPSEv94BP_Fyj85magOQiryue4tocTLNIIUKJj8XApHamOMGAW4oeGReG52vlYQ4HjZdiybAL7_gRd2wHdMDNHm0ODayznaO22jbnmB5oGLx_T4ooIH1nwk-fBEjIOYLF8fn2zgQOFVoj/s1600/thumbnails.data.bitmapdata_location.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="418" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiBPSEv94BP_Fyj85magOQiryue4tocTLNIIUKJj8XApHamOMGAW4oeGReG52vlYQ4HjZdiybAL7_gRd2wHdMDNHm0ODayznaO22jbnmB5oGLx_T4ooIH1nwk-fBEjIOYLF8fn2zgQOFVoj/s640/thumbnails.data.bitmapdata_location.PNG" width="640" /></a></div>
<br />
FTK will take us to the file offset. Once this has been done, we need to select the next 16384 bytes - the value from the thumbnails bitmapdata_length field. To do this, we can right click and choose "<b>Set selection length</b>":<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyANt5t0h9wGHquBLuusrE_J_QUn1xIrrzMnAxB6_2GyFVZ29dE3OadIIGfwRF9smBMCoFjddg1oNJV7ikX5OLlWlTF1xSHURelA2u-zuF5a8tOtUGg6nNOk3QKn7IAAjveYwZnrjVTP2V/s1600/screenshot.81.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="470" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyANt5t0h9wGHquBLuusrE_J_QUn1xIrrzMnAxB6_2GyFVZ29dE3OadIIGfwRF9smBMCoFjddg1oNJV7ikX5OLlWlTF1xSHURelA2u-zuF5a8tOtUGg6nNOk3QKn7IAAjveYwZnrjVTP2V/s640/screenshot.81.png" width="640" /></a></div>
<br />
And then fill in the value from the thumbnails bitmapdata_length field:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSTXurdr4RNPkLUdL4r9z23R5UTueMweaXVlAShOB-GK9gNFFaBQn0wVq8HnGfofOQqSIr9nC6Qpq5ssh0ctaSflBYy3H5pOvirGy5qBo3x9y85Xt0hd58bkR9F3I9OwCP5WgydpNzSgsk/s1600/selection_length.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="470" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSTXurdr4RNPkLUdL4r9z23R5UTueMweaXVlAShOB-GK9gNFFaBQn0wVq8HnGfofOQqSIr9nC6Qpq5ssh0ctaSflBYy3H5pOvirGy5qBo3x9y85Xt0hd58bkR9F3I9OwCP5WgydpNzSgsk/s640/selection_length.PNG" width="640" /></a></div>
<br />
<br />
Once this has been done, we can save the selection out to a file named "image.data":<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9Qz0shj17IUTK1KjwbYc0oM6B_OvKcykDHIg3km1gPyc7IqPxmgDHDBQqLGeNv4bEgM1kDmLR3T0Bt9QIUr3hPSUwh0aiYeIboFrsE7qxFatuCnMU9kpclqQlXScW6jK9vl9oeMno3l2O/s1600/ftk_save_selection.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="296" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9Qz0shj17IUTK1KjwbYc0oM6B_OvKcykDHIg3km1gPyc7IqPxmgDHDBQqLGeNv4bEgM1kDmLR3T0Bt9QIUr3hPSUwh0aiYeIboFrsE7qxFatuCnMU9kpclqQlXScW6jK9vl9oeMno3l2O/s400/ftk_save_selection.png" width="400" /></a></div>
<br />
Now that we have saved the bitmap out to a file - what do with do with it? Remember, it doesn't have a file header so just renaming it to .jpg or .png will not work. So how do we view it? <a href="https://www.gimp.org/downloads/">Gimp</a> - a free photo editing program has the ability to open up a raw bitmap and gives you the option to supply the width, height and image type.<br />
<br />
Use Gimp File > Open and select the Raw Image Data type. Opening up the image.data file presents the following dialog box:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgQOeDVhAzdjt2l0pr0ABWLKC91PTTZj5joyHCRt-802Dj-krBD2fRUEdG6BXcAv0J7XcT89R-a-EnpqcAYdXL8_1PZC34ZJQhQDUH-Xuag3U5l9K73cTNJB-okgiCLJKlMNOuvhyphenhyphen0eEyt/s1600/gimp_raw_bitmap.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjgQOeDVhAzdjt2l0pr0ABWLKC91PTTZj5joyHCRt-802Dj-krBD2fRUEdG6BXcAv0J7XcT89R-a-EnpqcAYdXL8_1PZC34ZJQhQDUH-Xuag3U5l9K73cTNJB-okgiCLJKlMNOuvhyphenhyphen0eEyt/s640/gimp_raw_bitmap.PNG" width="344" /></a></div>
<br />
Notice how the image looks all funky? That is because we have to specify the correct values to render the image. The image type is RGB Alpha (which I determined from monkeying around), and the width and height are 64 (which comes from the thumbnails table width and height) Once these are entered, the image displays correctly: <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNtUp0hMRxf61kLp1INFx1jGo9h6HqP1677w4zwddGbmkVyjDcYgqIO96QZXhiJPbZgthiCMrFx7DE9ekaXxS_IcjmYBGyWajKinbVUR0KZRYdJFnPntlgycZ3mJwQovl7AKoVI1Fwa1fZ/s1600/gimp_raw_bitmap_2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNtUp0hMRxf61kLp1INFx1jGo9h6HqP1677w4zwddGbmkVyjDcYgqIO96QZXhiJPbZgthiCMrFx7DE9ekaXxS_IcjmYBGyWajKinbVUR0KZRYdJFnPntlgycZ3mJwQovl7AKoVI1Fwa1fZ/s640/gimp_raw_bitmap_2.PNG" width="344" /></a></div>
<br />
<b>The Script</b><br />
Who wants to do this manually for each image? Like usual, python to the rescue. For this
particular script, I used a python library called Tkinter. This library
let me build a GUI app, in python, that works on multiple platforms! How
cool is that? The script also works from the command line as
well.<br />
<br />
<br />
To use the GUI, simply launch the python script with no commands:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVA1se2gG_XXZ3tmENwgWm34DZ8GPklkiVgWjTdyh6DhYi6olZr_wRArQzXd_J9ydsBPxiUg4wRL5tJ5NsdZBAtPZiykiKqOz_iQ2Cxm0nbIy5gV75lA6pJ-DmsUDFtUhkhPkAshPr91Xp/s1600/Windows_gui.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVA1se2gG_XXZ3tmENwgWm34DZ8GPklkiVgWjTdyh6DhYi6olZr_wRArQzXd_J9ydsBPxiUg4wRL5tJ5NsdZBAtPZiykiKqOz_iQ2Cxm0nbIy5gV75lA6pJ-DmsUDFtUhkhPkAshPr91Xp/s1600/Windows_gui.PNG" /></a></div>
<br />
Just to prove it works, here are screen shots of the same script working on Linux and Mac (tested on Ubuntu 14.4 and Mac OSX):<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcP5NmABjNEo0NLakWk17DKGPWn3G6TScBmN_wzwtjQFIpHMaieGWO55z6ptXV0CrkWqCTKBdAnkZdFvBKa7R57i8eVKaWly6cP4JYlEwdFBSativLbUH1cVnDOpiTZzJFq_hSaWuj9vdO/s1600/linux.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="201" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcP5NmABjNEo0NLakWk17DKGPWn3G6TScBmN_wzwtjQFIpHMaieGWO55z6ptXV0CrkWqCTKBdAnkZdFvBKa7R57i8eVKaWly6cP4JYlEwdFBSativLbUH1cVnDOpiTZzJFq_hSaWuj9vdO/s640/linux.png" width="640" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5lViVSrt2RQ_r_RgZsknmQXHzt7PAjBofy7A4Bql7CGo7IC8wpobiVnyrJfHWqdSEuwXXj_IoIuaDe9pgSuwL5AP3WgPcF6B-n90rgBqiN26s0KsGsj-zNKP8RWOb5uMvbqQ2q2WoKx7i/s1600/Mac_quicklook_parser.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="164" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5lViVSrt2RQ_r_RgZsknmQXHzt7PAjBofy7A4Bql7CGo7IC8wpobiVnyrJfHWqdSEuwXXj_IoIuaDe9pgSuwL5AP3WgPcF6B-n90rgBqiN26s0KsGsj-zNKP8RWOb5uMvbqQ2q2WoKx7i/s640/Mac_quicklook_parser.png" width="640" /></a></div>
<br />
<br />
Using the Gui is pretty easy - select the folder containing the com.apple.QuickLook.thumbnailcache, and a folder that will hold the outputs created by the script. The script will generate a report of the files and create a subfolder containing the images. The Excel option will generate an Excel spreadsheet with the images embedded in it.<br />
<br />
The command line syntax is as follows for tsv output:<br />
<br />
<div class="code">
python quicklook.py -d "C:\case\com.apple.Quicklook.thubmnailcache" -o "C:\case\quicklook_report"</div>
<br />
<br />
<br />
The command line syntax is as follows for Excel output:<br />
<br />
<div class="code">
python quicklook.py -d "C:\case\com.apple.Quicklook.thubmnailcache" -o "C:\case\quicklook_report" -t excel</div>
<br />
<br />
<br />
In order to use the script, the biplist and Pillow library needs to be installed. biplist is a python library for binary plist files, and Pillow is used to work with images. Both libraries are easy to install.<br />
<br />
To install biplist use easy_install:<br />
<br />
Linux/Mac: sudo easy_install biplist<br />
Windows: C:\<PYTHONDIR>\scripts\easy_install.exe biplist <br />
<br />
To install Pillow:<br />
<br />
Linux/Mac: sudo pip install Pillow<br />
Windows: C:\<PYTHONDIR>\scripts\pip.exe install Pillow<br />
<br />
The default output is TSV, however, if you would like an excel report the xlswriter python library needs to be installed:<br />
<br />
Linux/Mac: easy_install xlswriter<br />
Windows: C:\<PYTHONDIR>\scripts\easy_install.exe xlswriter <br />
<br />
I have also included a compiled Windows executable if you don't want to mess with installing all the libraries. <br />
<br />
<a href="https://github.com/mdegrazia/OSX-QuickLook-Parser">Download </a>the quicklook parserMari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com0tag:blogger.com,1999:blog-2456910508235577340.post-5750716872544200222016-09-22T07:34:00.001-07:002016-09-22T07:34:11.587-07:00Mac Live Imaging: Functionality Versus SpeedMy series on imaging a Mac would not be complete without covering how to do a live acquisition of a Mac. Now that FileVault2 appears to be the default during installs with Sierra, a live image may be very useful moving forward:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXKp5lqaSlh2FTiyry5T85fp6V0QO2D_4sOfWmyVTYOFxdc6cOLQ9mDmdDPQorbvxuDVsnYWELU8KydrBST_lUmtqjHmEPcc2Ae39ynZJ0AvTECBv5U4RSYd8iJjKcTjwJ9bvHcs3iE2IM/s1600/sarah_tweet.+PNG.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="115" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXKp5lqaSlh2FTiyry5T85fp6V0QO2D_4sOfWmyVTYOFxdc6cOLQ9mDmdDPQorbvxuDVsnYWELU8KydrBST_lUmtqjHmEPcc2Ae39ynZJ0AvTECBv5U4RSYd8iJjKcTjwJ9bvHcs3iE2IM/s400/sarah_tweet.+PNG.PNG" width="400" /></a></div>
<br />
If a hard drive is encrypted, a live image will allow you to create a logical image of the partition in an unencrypted state. In my previous posts I covered how to image a Mac using <a href="http://az4n6.blogspot.com/2016/07/how-to-image-mac-using-single-user-mode.html">single user mode</a> and a <a href="http://az4n6.blogspot.com/2016/06/how-to-image-mac-with-live-linux.html">Linux USB boot</a> disk. I've put off doing this blog post because there is a very detailed and well written <a href="http://www.505forensics.com/imaging-with-apple-ftkimager/">post</a> by Matt at 505Forensics that covers this topic. In his blog post, Matt walks though step by step how to image a Mac using the <a href="http://accessdata.com/product-download/digital-forensics/mac-os-10.5-and-10.6x-version-3.1.1">FTK Imager command line </a>tool for Mac OS X operating systems. As such, I wanted to cover how to do a live image using the <a href="https://en.wikipedia.org/wiki/Dd_(Unix)">dd command</a> as another option.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJMc8YuDm0-R-4ZAYekV4ddFs9ER9WvnHCaYQ7GYQw9gHaEAxgquI2iE1jusGvqoldrGwF1_47mTmOMNontuP8fmM84vBTBCJg6fpmQO0LxWar8BlcQNkLHOI0RzneYmb46FaUm7VWBF_V/s1600/live_disk_imaging.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="174" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJMc8YuDm0-R-4ZAYekV4ddFs9ER9WvnHCaYQ7GYQw9gHaEAxgquI2iE1jusGvqoldrGwF1_47mTmOMNontuP8fmM84vBTBCJg6fpmQO0LxWar8BlcQNkLHOI0RzneYmb46FaUm7VWBF_V/s320/live_disk_imaging.png" width="320" /></a></div>
Out in the field, I've found that it seems to take a longer time when using FTK Imager. I finally had a chance to do some testing and found that it took FTK Imager almost 2 hours to image a drive to a raw image (no compression). It took just 15 minutes using dd with an MD5. My test system was a MacBook Air, Early 2015, OS X El Capitan with a 75GB partition that was being imaged.<br />
<br />
Using FTK command line has some distinct advantages over dd. There are options to compress the image, choose e01 format and supply case information. However, if time and speed are an issue, dd may be a better option. For example, I've been onsite when 10 Macs needed to be imaged - dd was nice to use so we could finish up in time for dinner. If you can leave an image running overnight - it's probably not as critical. See below for the test data:<br />
<br />
FTK Imager: Total image time <b>1 hour, 49 min and 04 sec:</b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEisQ3hfjWPMg0Dk82z3KmuISkUhCT1BTOLU2NT5RvImuYLrPXHWx5qhStgmcG6PdVJdTDZeEpiEy0qUwdCdAw-NonDP5Dm9nWGNcbH1a_acXPwN23lsBfWLFAmqvCXfe-qW91EHKzJlLVGi/s1600/ftk-times.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEisQ3hfjWPMg0Dk82z3KmuISkUhCT1BTOLU2NT5RvImuYLrPXHWx5qhStgmcG6PdVJdTDZeEpiEy0qUwdCdAw-NonDP5Dm9nWGNcbH1a_acXPwN23lsBfWLFAmqvCXfe-qW91EHKzJlLVGi/s1600/ftk-times.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
dd image with md5: <b>15 minutes</b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaZHX4yxvF-QfJjYADutzh69Q_sr85CPtthDXHnkVhVWzJDLXWYtVs9l47oezIJx_A_gkzZnW-lhBAoI00Ts5BnS1XScjCCK4FMN4rxQsLpKWomgi5TkednyWA3kHHN9VOnDkxT5-kvgVL/s1600/dd-md5-time.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaZHX4yxvF-QfJjYADutzh69Q_sr85CPtthDXHnkVhVWzJDLXWYtVs9l47oezIJx_A_gkzZnW-lhBAoI00Ts5BnS1XScjCCK4FMN4rxQsLpKWomgi5TkednyWA3kHHN9VOnDkxT5-kvgVL/s1600/dd-md5-time.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
Please note - this testing is not by any means extensive (unlike the <a href="https://binaryforay.blogspot.com/2016/09/let-benchmarks-hit-floor-autopsy-vs.html">recent testing</a> by Eric Zimmerman on some forensic software). I created several images using both methods and the image times listed above were about the same.<br />
<br />
The first step is to run diskutil to see what the disk layout looks like and to determine what to image. I like to do this before I plug in my external USB. This makes it easier to see what drive needs to be imaged.<br />
<br />
<div class="code">
diskutil list</div>
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWj8p6-l8JqbuU5FIXYgESylmtWmcIKT8YG-dJLKVKOb2Sceq9cn8ZmlmoDo05RBEmIKWW-HzuKhKqa1WHLwTo-dgMEV-LqWglbhyWs1kInVhZyA1YZhdSDG1fS3DtTKgYWpsIiMJCSQjq/s1600/diskutil.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWj8p6-l8JqbuU5FIXYgESylmtWmcIKT8YG-dJLKVKOb2Sceq9cn8ZmlmoDo05RBEmIKWW-HzuKhKqa1WHLwTo-dgMEV-LqWglbhyWs1kInVhZyA1YZhdSDG1fS3DtTKgYWpsIiMJCSQjq/s1600/diskutil.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">No FileVault2/No Encryption</td></tr>
</tbody></table>
<br />
<br />
My system has both OS X and Windows (Bootcamp) installed. As you can see /dev/disk0 is my physical drive. Partition 2 is the Machintosh HD and Partition 4 is the Windows aka Bootcamp partition. The logical, active device I want to image is /dev/disk1. As you can see in the screenshot above, it is listed as the logical, unencrypted volume and refers back to disk0s2. (If you do run across a system with Bootcamp you will probably want to grab that partition as well, but for the purpose of this blog post I am focusing on the Mac partition)<br />
<br />
Below is a screen shot of what the same system looks like with FileVault2 turned on. Note that it says "Unlocked Encrypted". In this scenario, /dev/disk1 is logical volume I want to image.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5l4w4WA7aOER0h9prehYz4Uw4EFgPztoumakdaW9aW08SZqWx47ZsWOTegf_E_11xndCox5yIrl7MMgXRIoLsHzw8PqVZOkhDbuERMDoeIl3atHEZHJy7eQXXBZVSGRGBpvJEnSE027Gs/s1600/unecrypted_mac.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5l4w4WA7aOER0h9prehYz4Uw4EFgPztoumakdaW9aW08SZqWx47ZsWOTegf_E_11xndCox5yIrl7MMgXRIoLsHzw8PqVZOkhDbuERMDoeIl3atHEZHJy7eQXXBZVSGRGBpvJEnSE027Gs/s1600/unecrypted_mac.png" /></a></div>
<br />
<br />
<br />
<br />
Each /dev/disk has a corresponding /dev/rdisk: <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmXScbwxA2q6Eg1TrqFDuBSy-hYUZGlkYTrxXDMrIENcNoXA40Qo5bW12jQz6nM5U3U3DbRnl_wcsyIRrA0eqx8_kVqB8fqxaCbPacWtNKmd3d0I6dlWxlLiTmbaYbDWULkQgwpsOuSQXi/s1600/rdisk.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhmXScbwxA2q6Eg1TrqFDuBSy-hYUZGlkYTrxXDMrIENcNoXA40Qo5bW12jQz6nM5U3U3DbRnl_wcsyIRrA0eqx8_kVqB8fqxaCbPacWtNKmd3d0I6dlWxlLiTmbaYbDWULkQgwpsOuSQXi/s1600/rdisk.png" /></a></div>
<br />
rdisk is <a href="http://superuser.com/questions/631592/why-is-dev-rdisk-about-20-times-faster-than-dev-disk-in-mac-os-x">supposed</a> to be faster than /dev/disk. As such, we are going to use /dev/rdisk1 instead of /dev/disk1 in the dd command. <br />
<br />
Now would be a good time to plug in the external drive that will hold the image. On my system it auto mounted under /Volumes/<Device Name> <br />
<br />
For dd, I am going to use the syntax suggested by the <a href="http://www.forensicswiki.org/wiki/Dd">Forensic Wiki Page</a>. The syntax looks something like this:<br />
<br />
<div class="code">
sudo dd if=/dev/rdisk1 bs=4k conv=sync,noerror of=/Volumes/MAC-Images/my_image.dd </div>
<br />
<br />
Lets break down this command:<br />
<br />
<ul>
<li>sudo: run as super user </li>
<li>if=/dev/rdisk1: this stands for input file. This will be the disk that requires imaging</li>
<li>bs=4k : this is the block size used when creating an image. The Forensic Wiki recommends 4k</li>
<li>conv=sync,noerror: if there is an error, null fill the rest of the block; do not error out</li>
</ul>
<br />
Better yet - let's add in an MD5 so we can have a hash of the image to make it more "forensicky". In order to do this:<br />
<br />
<div class="code">
dd if=/dev/rdisk1 bs=4k conv=sync,noerror | tee /Volumes/MAC-Images/my-image.dd | md5 > /Volumes/MAC-Images/my-image-md5.txt</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-C6vVcYOUmskc97VPQWCnkbdscaQEtJzu9Nh6HtLAcGmNOc99KgJnnlNEQ_3QF09eCP4f05tUI2awo7W0sVSYeX4KhHaulN5Aky1qMWo_gpuNY7bWWus5j2buIQy1TTvhPiTvtgRqQwR7/s1600/dd-md5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-C6vVcYOUmskc97VPQWCnkbdscaQEtJzu9Nh6HtLAcGmNOc99KgJnnlNEQ_3QF09eCP4f05tUI2awo7W0sVSYeX4KhHaulN5Aky1qMWo_gpuNY7bWWus5j2buIQy1TTvhPiTvtgRqQwR7/s1600/dd-md5.png" /></a>
</div>
<br />
According to the <a href="https://en.wikipedia.org/wiki/Dd_(Unix)">forensic wiki</a>:<br />
<blockquote class="tr_bq">
"The above alternate imaging command uses dd to read the hard-drive being
imaged and outputs the data to tee. tee saves a copy of the data as
your image file and also outputs a copy of the data to md5sum. md5sum
calculates the hash which gets saved in mybgifile.md"
</blockquote>
<div class="separator" style="clear: both; text-align: center;">
</div>
Try not to fat finger the password like I did though...<br />
<br />
That's it! Happy imaging whichever tool you use.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com2tag:blogger.com,1999:blog-2456910508235577340.post-52932352569012793812016-09-03T08:06:00.002-07:002016-09-03T08:06:34.683-07:00Cookie Cruncher Update, Timelines, Chrome Parser and moreI just wanted to pass on that I had a chance to update my<a href="https://github.com/mdegrazia/Google-Analytic-Cookie-Cruncher"> Google Analytic Cookie Cruncher </a>to support Firefox up to version 48. I can't believe it's been two years since I've updated the code!<br />
<br />
I know I've said it before, but if you need me to update a tool to support a newer version of "X", please let me know - I'm happy to do so :) With everything else on my plate, I don't always have time to test each new browser for compatibility issues. Thanks to Heather Mahalik for reaching out to me with a student request to get it updated - sometimes I need that extra motivation.<br />
<br />
I also updated <a href="https://github.com/mdegrazia/Safari-Binary-Cookie-Parser">my script</a> that parses Google Analytics from Safari binary cookies. Mike O'Daniel reached out to me when the script crashed on him. Although he was unable to share the data due to privacy reasons, with a little back and forth trouble shooting we were able to determine what the issue was. He was parsing cookies from an iPad which contained URL encoded strings. None of my test data contained cookies formatted in this way and I did not have access to an iPad. Once the issue was fixed in the script he was off an running. Thanks to Mike for reaching out to me to let me know that there were issues, and taking the time to help trouble shoot it since I was not able to replicate the issue. <br />
<br />
I also wanted to push out a <a href="https://github.com/mdegrazia/Chrome-Parse">simple little parser </a>for Chrome Internet History and Downloads. I recently spoke at the <a href="http://htciaconference.org/">HTCIA conference</a> about mini-timelines (and even micro timelines). While this concept is nothing new, I have found this process to be invaluable during the cases I work. Harlan has <a href="http://windowsir.blogspot.com/2015/04/micro-mini-timelines.html">blogged</a> many times about the process and advantages of it, so I won't go into detail here. For the lab I taught, I just needed to output some basic Chrome Internet History into TLN format so I wrote a <a href="https://github.com/mdegrazia/Chrome-Parse">Chrome parser</a> in python.<br />
<br />
Now this tool does not show every single thing that is available in the Chrome History. I just stuck to the basic information: Visit time, URL, Hit Count etc. Sometimes too much information can cloud the timeline, making it difficult to pick out patterns of activity, or create so much noise the next lead gets lost in all the output.<br />
<br />
I like the data in my timeline to be concise and clear. It reminds me a little of keyword searching. If the term is vague, you may be casting a wider net, but relative results could get buried in a million hits. It's going to take a lot of sifting to find that golden nugget. However, if you use a carefully crafted keyword, you can focus in on what it is you are looking for. Timelines are the same. Carefully picking the artifacts you want to add in to the timeline can help you hone in on relevant data quickly.<br />
<br />
The other thing I wanted to discuss was Volatility plugins. I recently had the chance to run through a demo at a <a href="https://www.meetup.com/Tucson-Python-Meetup/events/233454473/">Python Meetup group </a>on what Memory forensics is, and how <a href="http://www.volatilityfoundation.org/">Volatility</a> can be used to analyze memory. As part of this, I "wrote" my first volatility plugin. Now, I say "wrote" because it was really just modifying a couple of lines in someone else's code to do something a little different.<br />
<br />
Volatility has provided a<a href="https://github.com/volatilityfoundation/volatility/wiki/Command-Reference-Registry-Api"> nice interface</a> to grab various keys from the registry. In fact, it reminds me of the way plugins are handled in <a href="https://github.com/keydet89/RegRipper2.8">RegRipper.</a> If there is a key that you want that is not currently supported, look for a plugin that is similar and see if you can tweak it. It's a great way to start out, and as you tweak more and add a little bit here and there, you being to understand how things work.<br />
<br />
I just started with something simple - pulling the computer name. This is just one key, with no binary data to convert: <br />
<br />
HKLM\CurrentControlSet001\Control\ComputerName\ComputerName<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUXu_ipS3KdSp-sNub45ovGmIvGmHXkJvUaB2dHVicXtxQmwNqfjY7uSkFBWOA5GpTeZ_ABxtBx-6FB70gd_uUL9QBkdxAkK8aUeCg56TczmODl8Wthe7X9TJdhBcDeuyO6daSdX-BSX6Y/s1600/computer_registry_key.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="377" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUXu_ipS3KdSp-sNub45ovGmIvGmHXkJvUaB2dHVicXtxQmwNqfjY7uSkFBWOA5GpTeZ_ABxtBx-6FB70gd_uUL9QBkdxAkK8aUeCg56TczmODl8Wthe7X9TJdhBcDeuyO6daSdX-BSX6Y/s640/computer_registry_key.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
I found another Volatility plugin that pulls a key from the system hive, <a href="https://github.com/volatilityfoundation/volatility/blob/master/volatility/plugins/registry/shutdown.py">shutdown.py</a> - changed a few lines of code, and et voila! My first plugin. Ok - nothing earth shattering or difficult, but it's the first step in understanding how things work. That's often the way that I write many of my scripts - break it down into pieces, find code examples, and put it all together. Pretty soon I actually remember some of it, and my skill set advances.<br />
<br />
The <a href="https://github.com/volatilityfoundation/volatility/blob/master/volatility/plugins/registry/shutdown.py">original code </a>was written by Jamie Levy (@gleeda), and pulls the shutdown time from the registry. Below is a example of what I did. I just commented out what I didn't need, and modified what I did need.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5r8UfOYfnlzabZbQ20oSZu-Wr8bk2GjohKFQfdQaqtAyNSX6CWirtENmJ77TJDXdgZtzl9DBvnwvsmoilxFv78kMmcovAQmjfoBscefrtFcycU-4Alwp7E2PaUyd5d9jsSIqkj64OJ70u/s1600/plugin.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="340" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5r8UfOYfnlzabZbQ20oSZu-Wr8bk2GjohKFQfdQaqtAyNSX6CWirtENmJ77TJDXdgZtzl9DBvnwvsmoilxFv78kMmcovAQmjfoBscefrtFcycU-4Alwp7E2PaUyd5d9jsSIqkj64OJ70u/s640/plugin.png" width="640" /></a></div>
<br />
While it may not be complex, it gets the job done and I learned something new in the process. Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com0tag:blogger.com,1999:blog-2456910508235577340.post-86096680677790931872016-07-19T08:58:00.002-07:002016-07-19T08:59:28.568-07:00Mounting and Reimaging an Encrypted FileVault2 Mac Image in Linux<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtJNxPmISWQ6M3jA65pb5Mm6wXZOlrop7aWhxNIdOsRerWsDTTlTd31U-4rXmdKM0n9Zr2FOKJtlV0S7UrD0uO83_fgkpQk75lbSH762zaSDxUr9WYA0KMoGNUCAynlL5yw7OWdFgVQUiN/s1600/tux_filevault.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtJNxPmISWQ6M3jA65pb5Mm6wXZOlrop7aWhxNIdOsRerWsDTTlTd31U-4rXmdKM0n9Zr2FOKJtlV0S7UrD0uO83_fgkpQk75lbSH762zaSDxUr9WYA0KMoGNUCAynlL5yw7OWdFgVQUiN/s200/tux_filevault.png" width="180" /></a>Before I continue my <a href="http://az4n6.blogspot.com/2016/07/how-to-image-mac-using-single-user-mode.html" target="_blank">series</a> on how to image Mac systems, I wanted to cover how to mount and work with FileVault2 encrypted Mac images. By "work with", I mean decrypt it and create an image of the decrypted volume in either raw (dd) or E01 format to pull into X-Ways, EnCase etc. To do this three things are needed:<br />
<br />
1) A full disk image of the encrypted system in raw format (dd)<span style="background-color: yellow;"> </span><br />
2) The<a href="http://digital-forensics.sans.org/community/downloads" target="_blank"> SIFT Workstation</a> - it has all the (free!) tools needed already installed <br />
3) The password or recovery key for the volume.<br />
<br />
<br />
<br />
For this example, I am going to use the encrypted disk image of a Mac I created from this <a href="http://az4n6.blogspot.com/2016/07/how-to-image-mac-using-single-user-mode.html" target="_blank">previous turotiral</a>. Below is what the encrypted image looks like in FTK Imager. Note that the second partition, MacOSX, is showing as an Unrecognized file system. This is because it is encrypted with FileVault2:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6KkFideOzGSlLfDRGlmpBEcffSZZv4Tvv9odNmX6iRXsOtA13U07MfKIr94w26W8cY1EoR9okDgHYCQE8Bl8_bY5yyxnXQotqhVZLMz5D4pyXBQDbdagQDNzsSJLDASaEe8Ey71B8ouvJ/s1600/ftk_encrypted.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="289" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6KkFideOzGSlLfDRGlmpBEcffSZZv4Tvv9odNmX6iRXsOtA13U07MfKIr94w26W8cY1EoR9okDgHYCQE8Bl8_bY5yyxnXQotqhVZLMz5D4pyXBQDbdagQDNzsSJLDASaEe8Ey71B8ouvJ/s640/ftk_encrypted.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
Another way to verify that the partition is encrypted is to look for the EncryptedRoot.plist.wipekey on the Recovery partition. In fact, we are going to need this to decrypt the drive, so I am just going to export out this file while I have it opened in FTK Imager. Mine was located under Recovery HD
[HFS+]\Recovery
HD\com.apple.boot.P\System\Library\Caches\com.apple.corestorage\EncryptedRoot.plist.wipekey:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi203gMX8Z3NNhYa3WxBPU-PkoGafoiuIbM0f2yGhAItOnD3BZjOwYirW9H68ku5GzuePY34sZISPq8nDBb3db1S6qVnxuQPl0EbHFGf-FGwM9ZEc1i3vNRFx0nyRS-9FLhUACfasKC-Qv5/s1600/EncryptedRoot.plist.wipekey.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi203gMX8Z3NNhYa3WxBPU-PkoGafoiuIbM0f2yGhAItOnD3BZjOwYirW9H68ku5GzuePY34sZISPq8nDBb3db1S6qVnxuQPl0EbHFGf-FGwM9ZEc1i3vNRFx0nyRS-9FLhUACfasKC-Qv5/s640/EncryptedRoot.plist.wipekey.jpg" width="640" /></a></div>
<br />
<br />
<br />
If you're using SIFT in a VM, the first step is to create a shared folder(s) for where the image is located, and where you want your decrypted dd/E01 image to go. Here I have two USB drives shared as E: and G:. The E: drive contains my encrypted image and my EncryptedRoot.plist.wipekey file. The G: drive is where I am going to dump the unencrypted image. In <a href="https://www.virtualbox.org/wiki/Downloads" target="_blank">Virtual Box</a> these settings were located under Settings > Shared Folders.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhirU9QaLt4ToMkbn-lzdBtt0UMw2c-_rpe3xKuS_nLpBBEBhjiJzl_u4X-z3MYPieDUr28739lzb9NfHKleAQLAP9Yx6ymJuCMM8kpwY8uK_SKIJbqcGoj5MuKClyHc2StPn8OHUBTtO2q/s1600/VM+Shared+folder.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="419" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhirU9QaLt4ToMkbn-lzdBtt0UMw2c-_rpe3xKuS_nLpBBEBhjiJzl_u4X-z3MYPieDUr28739lzb9NfHKleAQLAP9Yx6ymJuCMM8kpwY8uK_SKIJbqcGoj5MuKClyHc2StPn8OHUBTtO2q/s640/VM+Shared+folder.PNG" width="640" /></a></div>
<br />
Next, I am going to make a mount point to mount the image:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpXraAxJwN_SyhdwChTQiTNidfmrksiC5N2PP_aF0Yov6rRLOc87LdJxTbH8mmlXX3jcuJx1TxU6APQkYgACzz4p3bsq1JFwn6gYmcKXgm0DOh8K0mS0WQFzOkNy8i6VHZzySLdzBouYip/s1600/mnt_Mac.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgpXraAxJwN_SyhdwChTQiTNidfmrksiC5N2PP_aF0Yov6rRLOc87LdJxTbH8mmlXX3jcuJx1TxU6APQkYgACzz4p3bsq1JFwn6gYmcKXgm0DOh8K0mS0WQFzOkNy8i6VHZzySLdzBouYip/s1600/mnt_Mac.PNG" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
Now I am going to change into the directory where I have my image and wipekey:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLDFsoAJ8OAnBjQps4UnTxNOnFUsGAuz0-7Lx9rYpnCrOcDKFfdu7AAN6oQGSvDbbTv_vY3hSatyCw0ZVwd1VKFm7wDLKu8AYc7sK3JN9Sb5YoI2iJV3lIS5EB-NzA69zyihp-joNXzSKO/s1600/ls_E_Drive.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="130" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLDFsoAJ8OAnBjQps4UnTxNOnFUsGAuz0-7Lx9rYpnCrOcDKFfdu7AAN6oQGSvDbbTv_vY3hSatyCw0ZVwd1VKFm7wDLKu8AYc7sK3JN9Sb5YoI2iJV3lIS5EB-NzA69zyihp-joNXzSKO/s640/ls_E_Drive.PNG" width="640" /></a></div>
<br />
Joachim Metz has written a library, <a href="https://github.com/libyal/libfvde" target="_blank">libfvde</a>, to access FileVault encrypted volumes. I will be using fvdemount from this library to mount the encrypted partition. He has excellent documentation on his <a href="https://github.com/libyal/libfvde/wiki" target="_blank">wiki </a> - I will pretty much be following that in the steps below. <br />
<br />
<br />
I need to get the partition offset in bytes to pass to fvdemount. <a href="http://www.sleuthkit.org/sleuthkit/man/mmls.html" target="_blank">mmls </a>from the Sleuth Kit can be used to get the offset from the image:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidWVyktsyBlLxBuFtw6Mlw6L-xw91DsMIlGSFhohp5_aZhKU17l_uHSC9w898m0nnTm5MIaCiQImsVapHbV6oelyAjRQihR9MHIXetQuYGpVmlV23YN-VxZlrGG9vNxeFOI6zm1WTFQbHQ/s1600/mmls_sleuth_kit.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidWVyktsyBlLxBuFtw6Mlw6L-xw91DsMIlGSFhohp5_aZhKU17l_uHSC9w898m0nnTm5MIaCiQImsVapHbV6oelyAjRQihR9MHIXetQuYGpVmlV23YN-VxZlrGG9vNxeFOI6zm1WTFQbHQ/s640/mmls_sleuth_kit.PNG" width="640" /></a></div>
<br />
According to output above, the MacOSX encrypted partition starts at sector 409640. To get the offset in bytes multiply the offset (409640) times 512 bytes / per sector. I will need to pass this offset (-o), the EncryptedRoot.plist.wipekey (-e),the password or recovery key (-p), my image and the mountpoint to fvdemount:<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhn_GbicnvE2RJYHEBK7lqC9VDR6kC-0tMcU1M9HwI5yx-TXoOnyRlAad61nNYCnhRf8-z5xrF2NBnvR72JrBpuXP69jmu1Yh-eBNIaFHlUMvkjhMj_G2R_4EEgsmWjHzTuuZUbjwgEytww/s1600/mount_filevault_fvdemount.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="316" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhn_GbicnvE2RJYHEBK7lqC9VDR6kC-0tMcU1M9HwI5yx-TXoOnyRlAad61nNYCnhRf8-z5xrF2NBnvR72JrBpuXP69jmu1Yh-eBNIaFHlUMvkjhMj_G2R_4EEgsmWjHzTuuZUbjwgEytww/s640/mount_filevault_fvdemount.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
fvdemount will create a device file named "fvde1" under the mount point. A quick file command confirms it is the HFS volume:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4psJSrgiUwbozSS8fUlj1iWvNNEcnZHjorozdY1cUs4-ZB9ETqhce7CkLOF_WgzJilSzgYH3qulUkCGWEi9CRMp3yRXYFzTXoTix2XU_LCm74ycTNmV-R_Q-kMZ3o884uUDMs0UiZVlf4/s1600/file_fvde1.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="194" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4psJSrgiUwbozSS8fUlj1iWvNNEcnZHjorozdY1cUs4-ZB9ETqhce7CkLOF_WgzJilSzgYH3qulUkCGWEi9CRMp3yRXYFzTXoTix2XU_LCm74ycTNmV-R_Q-kMZ3o884uUDMs0UiZVlf4/s640/file_fvde1.PNG" width="640" /></a></div>
<br />
To further verify everything is unencrypted, fvde1 can be mounted as a loopback device to show the files system:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZwuoPkpqEM7qFpI1y9heSoW15rVhrKj1goZ3wgOEI9Zv-hoQA6LsTVQOpWdwSn6aqYDEBA8esO8wQTa5H0VfI0hwipwGWkhVCPOp7yEAyu9MMgilaNSGihCKc5zTV8nvBH_iGxxE8dLtW/s1600/ls_mnt_mac_filesystem.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="156" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZwuoPkpqEM7qFpI1y9heSoW15rVhrKj1goZ3wgOEI9Zv-hoQA6LsTVQOpWdwSn6aqYDEBA8esO8wQTa5H0VfI0hwipwGWkhVCPOp7yEAyu9MMgilaNSGihCKc5zTV8nvBH_iGxxE8dLtW/s640/ls_mnt_mac_filesystem.PNG" width="640" /></a></div>
<br />
As shown above, I can now see the unencrypted Mac partition.<br />
<br />
If your preference is to work with an image under Windows with tools like X-Ways, EnCase etc, an image can be taken of the unencrypted device, /mnt/Mac/fvde1.<br />
<br />
For E01 format, <a href="http://linux.die.net/man/1/ewfacquire" target="_blank">ewfaquire </a>can be used:<br />
<br />
<div class="code">
ewfaquire /mnt/Mac/fvde1</div>
<br />
<br />
For raw (dd) format the following syntax can be used. I like to have a progress bar so I am using using the<a href="http://linux.die.net/man/1/pv" target="_blank"> pv</a> command with dd. For dd, I am using the recommend parameters from the <a href="http://www.forensicswiki.org/wiki/Dd" target="_blank">Forensic Wiki</a>.<br />
<br />
<div class="code">
dd if=/mnt/Mac/fvde1 bs=4K conv=noerror,sync | pv -s 999345127424 | dd of=/media/sf_G_Drive/Image/Mac_fvdemount_unencrypted.dd</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLC9C-6x7625AIrT-9u8TXUKL9hUFe_KJ227p958BiEiupiUul00DgXg0IkEvBz0fFAhbf4GgxUASZkhN6Ip5nMqfXzAkUByl-G59ATjfeXvulCexf357qG6aQi325Dd-rQaGHe6t84cpD/s1600/dd_pv.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="138" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLC9C-6x7625AIrT-9u8TXUKL9hUFe_KJ227p958BiEiupiUul00DgXg0IkEvBz0fFAhbf4GgxUASZkhN6Ip5nMqfXzAkUByl-G59ATjfeXvulCexf357qG6aQi325Dd-rQaGHe6t84cpD/s640/dd_pv.PNG" width="640" /></a></div>
<br />
<br />
-s is the size, which can be taken from the length of the partition, 1951845925 sectors * 512 bytes/sector = 999345113600 bytes (aprox. 1TB)<br />
<br />
After the image completes, it can now be be opened and viewed in all it's unencrypted glory in the tool of your choice:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEzg7KDNc-wloIQpU2ULCUVE3kg9SqFwGlqeeUfYDEGYyoqzs4caq7S5w3qWz-uLqG72zIyz1VPDBSoswjL0NqqN94aJXCYcQp419R2ferqXzs_9q-UV8KuEZ_mBtlCoIn_xkTEXBxk2nj/s1600/ftk_image.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="436" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEzg7KDNc-wloIQpU2ULCUVE3kg9SqFwGlqeeUfYDEGYyoqzs4caq7S5w3qWz-uLqG72zIyz1VPDBSoswjL0NqqN94aJXCYcQp419R2ferqXzs_9q-UV8KuEZ_mBtlCoIn_xkTEXBxk2nj/s640/ftk_image.PNG" width="640" /></a></div>
<br />
A Mac system can also be used to mount an encrypted volume. I may write a post about that at a later time. I know not all examiners have access to a Mac system, so I wanted to focus on this method first. Plus, I like good 'Ol Tux.<br />
<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com4tag:blogger.com,1999:blog-2456910508235577340.post-77507826710538379712016-07-06T22:04:00.000-07:002016-08-01T23:10:09.570-07:00How to image a Mac using Single User Mode<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6H9slMIuo2oJMrnRduo1n46Wvt52uUjzpmPy5eubZH_Bv-JmnArZsZTd8DL13QrsM_bQjP6lYTd8D57T1GRZpb1jG2VMu2jpNgbGgf0Ev5UJCB08D5-xK7xoGJK68ctZzrwB4KEkC21mV/s1600/i_am_root.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="254" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6H9slMIuo2oJMrnRduo1n46Wvt52uUjzpmPy5eubZH_Bv-JmnArZsZTd8DL13QrsM_bQjP6lYTd8D57T1GRZpb1jG2VMu2jpNgbGgf0Ev5UJCB08D5-xK7xoGJK68ctZzrwB4KEkC21mV/s320/i_am_root.png" width="320" /></a></div>
This is the second post in my series on different ways to image a Mac. My <a href="http://az4n6.blogspot.com/2016/06/how-to-image-mac-with-live-linux.html" target="_blank">first post</a> was on how to image a Mac with a bootable Linux distro. This post will cover another option, creating an image by booting a Mac into single-user mode. I plan on following up this post with posts on creating a live image and how to mount and work with FileVault encryption after an image is complete.<br />
<br />
Single-user mode is a limited shell that a Mac can boot into before fully loading the operating system. In single-user mode, the internal hard drive is mounted read only and a limited set of commands are available. Once in single-user mode, a USB drive can be attached and dd can be used to create an image.<br />
<br />
In order to mount the USB drive, the internal drive needs to be changed to read/write to create a mount point. While not as forensically sound as using a write blocker or booting into a Linux distro, less changes are made than fully booting the operating system to take a live image. This may be a good option where it is acceptable to get a live image, but the examiner wishes to minimize changes to the hard drive. Another benefit is that if there is FileVault encryption, the encrypted drive is decrypted after a username and password are supplied.<br />
<br />
The system I used for testing was a Mac Mini, OS X Version 10.8.5 with one hard drive. Three partitions
were created by default during the initial setup: an EFI partition, a MacOSX partition, and a recovery partition.<br />
<br />
I tested two scenarios, one without encryption and one with encryption (FileVault 2). For each step I will cover both scenarios. The high level steps are:<br />
<br />
1) Boot into single-user mode <br />
2) Determine the disk to image<br />
3) Mount the USB drive that will hold the image<br />
4) Run the dd command to create the image<br />
<br />
[Edit 8/1/2016] Please read the comments as well. I have had some people in the community provide some great tips and suggestions since this was posted! <br />
<br />
<span style="color: red;"></span><u><b>Step 1 - Boot into single-user mode</b></u><br />
The first step is to boot into single-user mode. While the system is booting, select COMMAND-S to enter single-user mode. I usually hold down this key combo before I even power on the system so I don't accidentally "miss" it. At this time, I do not to have the USB drive that will hold the image plugged in.<br />
<br />
<b>Unencrypted</b><br />
If the system is not encrypted a bunch of white text will scroll and finally present a shell with root:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkWIaBAFoIUnXUfuoA-GPmAY_KWl2gNOXM6nIIirBc1tv2mCZaB4lgOwIn3ROzsMt-zMudNdjEO1pERJSCQeoAbOOkAddFdHiWH9cWz5zfWJLaxlatmeL6ZHh90brJOUYe2MyoI4_sGPMO/s1600/mac_single_user_mode.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="392" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkWIaBAFoIUnXUfuoA-GPmAY_KWl2gNOXM6nIIirBc1tv2mCZaB4lgOwIn3ROzsMt-zMudNdjEO1pERJSCQeoAbOOkAddFdHiWH9cWz5zfWJLaxlatmeL6ZHh90brJOUYe2MyoI4_sGPMO/s640/mac_single_user_mode.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<b>Encrypted</b><br />
If the system is encrypted, some text will fly by that says efiboot, and then a GUI window will pop up asking for the username and password:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj0IYnq6o-NpPwK7uU4RRzCxGhbBFAnAb0Ri2l1zVlxZiY9v-BPElmcRoUbK5nKn0zoqLkm-pOjt6JpWHjQqA6J5f9rbC1I4_9p3aOssQhHUu5DRAOMEc0N1Zg76VDQpIXtTr6UboBzBQL/s1600/efi_boot_screen.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="174" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjj0IYnq6o-NpPwK7uU4RRzCxGhbBFAnAb0Ri2l1zVlxZiY9v-BPElmcRoUbK5nKn0zoqLkm-pOjt6JpWHjQqA6J5f9rbC1I4_9p3aOssQhHUu5DRAOMEc0N1Zg76VDQpIXtTr6UboBzBQL/s640/efi_boot_screen.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihCM3peovuFF60vTv9NshcQjm7P-AR-ToLyIKJBsPFEonI9lvjvPVPKcN8rgtPa-bXr4HI9AohbyJ4y-xC-8y3UfnpnfewPXMZSzLiXb2LZMaAhDYaq9KyfY2H9SdtcuGn9YlRQoL68b9v/s1600/gui_login.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="267" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihCM3peovuFF60vTv9NshcQjm7P-AR-ToLyIKJBsPFEonI9lvjvPVPKcN8rgtPa-bXr4HI9AohbyJ4y-xC-8y3UfnpnfewPXMZSzLiXb2LZMaAhDYaq9KyfY2H9SdtcuGn9YlRQoL68b9v/s320/gui_login.png" width="320" /></a></div>
<br />
After the username and password are entered, the single-user boot process continues and drops into a shell similar to the unencrypted system.<br />
<br />
<u><b>Step 2 - Determine what to image</b></u><br />
The next step is to determine what block device to copy for the dd command. In order to determine this, use the ls command to get a list of the available disks under the /dev directory. As I mentioned before, I prefer to do this before I plug the USB drive in so I don't have to try and guess which is the internal hard drive and which is the USB drive. (OS X has a disk utility called diskutil that presents more verbose
information about the disks, however, it is not available in single-user
mode) <br />
<br />
<div class="code">
ls /dev/disk* </div>
<br />
<br />
The output is slightly different between the encrypted and unencrypted drive, which I discuss below. <br />
<br />
<b>Unencrypted Drive</b><br />
On the test unencrypted system there is one disk, disk0, with three partitions: disk0s1, <br />
disk0s2, and disk0s3. For this particular system, the image should be of /dev/disk0: <br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh10VX-EHsjVN_e-1kQtLtisAeyq2qftbtc3YTb3Kd2XlspEwufOvlg6UDQrN6bn0E8h0j0n-vkNwaaLYDpCRu7hmqPMuhfzxKwc5gPBFSkGaAvPaVSeMMw_JUOhUTMb9j3R9whwNcciwVK/s1600/ls_dev_disk_unencrypted.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="68" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh10VX-EHsjVN_e-1kQtLtisAeyq2qftbtc3YTb3Kd2XlspEwufOvlg6UDQrN6bn0E8h0j0n-vkNwaaLYDpCRu7hmqPMuhfzxKwc5gPBFSkGaAvPaVSeMMw_JUOhUTMb9j3R9whwNcciwVK/s640/ls_dev_disk_unencrypted.png" width="640" /></a></div>
<br />
<br />
<b>Encrypted Drive</b><br />
Note the addition of the /dev/disk1 on the encrypted system:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6pdEnAiV2M6BP0xf1hvWN8lwA-sQVWaUMbLBG0kJPHQ5knt1NJWXC30jEPSvQJGwYv3qpKNFznyYq6qpdwd1uQJ5dP_opD09oxb1QVUbO6r3ZL-uOuSC-d19-3QJaXf4NRZd98BIW04kn/s1600/encrypted_ls_dev.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="89" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi6pdEnAiV2M6BP0xf1hvWN8lwA-sQVWaUMbLBG0kJPHQ5knt1NJWXC30jEPSvQJGwYv3qpKNFznyYq6qpdwd1uQJ5dP_opD09oxb1QVUbO6r3ZL-uOuSC-d19-3QJaXf4NRZd98BIW04kn/s640/encrypted_ls_dev.png" width="640" /></a></div>
<br />
What is this /dev/disk1? Using file -sL on each partition can give a little bit more insight into what is going on. (Note - I ran these commands while in a terminal because there was no
good way for me to get a screen shot in single-user mode...the text went
all the way across the screen. However, the commands and outputs are similar while in single-user mode)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpZL9jh_iYAt68jBwGwXu0yKR9KdV5NHgGKFxZPjXWAQL59iPc6KwGKbM3r8q0QGS3eearYWo7GqYMTDJ6ZDDkST3b3rAKum7SDtXnc2wIqoTFlafMIdb8CmvSETvN4f1kkPqFu5ugVTho/s1600/file_dev_disk0.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjpZL9jh_iYAt68jBwGwXu0yKR9KdV5NHgGKFxZPjXWAQL59iPc6KwGKbM3r8q0QGS3eearYWo7GqYMTDJ6ZDDkST3b3rAKum7SDtXnc2wIqoTFlafMIdb8CmvSETvN4f1kkPqFu5ugVTho/s1600/file_dev_disk0.png" /></a></div>
<br />
From these results I can tell that disk0s1 is the EFI partition, and disk0s3 is an HFS partition. disk0s2 is showing as "data". This happens when the file command can't tell what the file is, it just gives a generic "data" in response - which makes sense if the partition is encrypted.<br />
<br />
Some <strike>quick</strike> math give us the partition sizes:<br />
<br />
EFI disk0s1 size = 409600 sectors X 512 bytes per sector = 209715200 bytes = ~210 MB<br />
HFS disk0s3 size = 4096 bytes per block X 158692 blocks = 650002432 bytes = ~650 MB<br />
<br />
Next, I want to see what size disk0s2 is. I can use fdisk /dev/disk0s2 for this:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-AzoyFcf_JiaiOy6VyJn56tld1A_NPKqIr_nYKeKGV_ycvjKRvuYb_jStVvCu74y_SQ9srOwE7YzfbqrVhn_akFdD_JD2PopuIiZCaPZrcsGrw2oWPBSIb4A6hra2GcL_ePRvyjqvgxui/s1600/fdisk.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-AzoyFcf_JiaiOy6VyJn56tld1A_NPKqIr_nYKeKGV_ycvjKRvuYb_jStVvCu74y_SQ9srOwE7YzfbqrVhn_akFdD_JD2PopuIiZCaPZrcsGrw2oWPBSIb4A6hra2GcL_ePRvyjqvgxui/s1600/fdisk.png" /></a></div>
<br />
disk0s2 size = 1951845952 sectors X 512 bytes per sector = 999345127424 bytes =~999.3 GB. Definitely the biggest of them all!<br />
<br />
Now I want to see how big /dev/disk1 is to compare it to the other partitions. Here I will use /dev/rdisk1 because /dev/disk1 is busy. /dev/rdisk is the <a href="http://superuser.com/questions/631592/why-is-dev-rdisk-about-20-times-faster-than-dev-disk-in-mac-os-x" target="_blank">raw disk</a> of /dev/disk1:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihHL-9-gca4eQmfw0G-uli24pDn0AYX46eWGcQlYy3VWygZFX6zcsq_n9hvALSdr-J-gGIM__gf97S3PbWUflOcHvSZbBDL-6itzB0m_8HU3hOF0sRplMfGaK_COLalgHnMFrkhgtu17i0/s1600/file_rdisk1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="176" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihHL-9-gca4eQmfw0G-uli24pDn0AYX46eWGcQlYy3VWygZFX6zcsq_n9hvALSdr-J-gGIM__gf97S3PbWUflOcHvSZbBDL-6itzB0m_8HU3hOF0sRplMfGaK_COLalgHnMFrkhgtu17i0/s640/file_rdisk1.png" width="640" /></a></div>
rdisk1 size = 4096 bytes per block X 243898823 blocks = 999009579008 bytes =~ 999 GB<br />
<br />
/dev/disk0s2 and /dev/disk1 are about the same size, 999GB, and /dev/disk1 is a readable HFS partition. Based on my experience and the outputs above, it appears /dev/disk1 is the OS X partition (disk0s2) in a decrypted state.<br />
<br />
For imaging, either /dev/disk0 or /dev/disk1 can be used. If /dev/disk0 is used, all three partitions will be captured, but the data in the MacOSX partition - /dev/disk0s2 will remain in the
encrypted state. If /dev/disk1 is imaged, it will have the MacOSX data in an
decrypted state,but will not have partition 1 (EFI partition) and partition 3 (Recovery partition). I like to grab both /dev/disk0 and /dev/disk1.<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<u><b>Step 3 - Mount the external USB Drive </b></u><br />
The next step is to mount the external USB drive so the image can be saved onto it. The USB drive can be formatted in FAT32 or HFS. FAT32 has the benefit of both Windows and Mac being able to access it, but it has a 4GB file size limit. While HFS does not have the 4GB limit, Windows is not able to see it by default (if you have a Mac with bootcamp your Windows OS should be able to read HFS if the bootcamp drivers are installed).<br />
<br />
For my tests I used a FAT32 USB drive for the unencrypted system, and an HFS USB drive for the encrypted system so I could demonstrate the syntax for both.<br />
<br />
After plugging in the USB drive, run ls /dev/disk* again. Compare the outputs to determine which /dev device belongs to the USB drive<br />
<br />
<div class="code">
ls /dev/disk*</div>
<br />
<br />
<b>Unencrypted</b><br />
For this system the FAT32 USB drive has been inserted, which shows up as /dev/disk1. The partition that needs to be mounted is /dev/disk1s1:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjx3_MlmSTaGjChK4QMQbG68rGqV7AZ1Da1S6tBjtbSkbL0T-FtjrrHj6zCwk8aOjnJ5NOM6fMX0suBYta7JILoo8bndFtOSYYrqPFPSK5Z-SbNutDH-lE00gWTAdr9qNHfa_YgnHFFqW8P/s1600/ls_dev_disk_usb_unencrypted.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="130" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjx3_MlmSTaGjChK4QMQbG68rGqV7AZ1Da1S6tBjtbSkbL0T-FtjrrHj6zCwk8aOjnJ5NOM6fMX0suBYta7JILoo8bndFtOSYYrqPFPSK5Z-SbNutDH-lE00gWTAdr9qNHfa_YgnHFFqW8P/s640/ls_dev_disk_usb_unencrypted.png" width="640" /></a></div>
<br />
<b>Encrypted</b> <br />
For this system the HFS USB drive has been inserted, which shows up as /dev/disk2. This drive has two partitions. The partition that needs to be mounted is /dev/disk2s2:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLf-DhDpG3KiklrEokiC8249uFGcNntoFxsMpdz1a1tG243Fce2qZgpZNWaXofzd8qdLwtCoJwCxyKQQWGxHNrxhoPZHsLoFUI0GBgZ-0H3D74hSxCG1CgH4LJffk4FFj17FE4ftxBNLVr/s1600/dev_disk_2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="130" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLf-DhDpG3KiklrEokiC8249uFGcNntoFxsMpdz1a1tG243Fce2qZgpZNWaXofzd8qdLwtCoJwCxyKQQWGxHNrxhoPZHsLoFUI0GBgZ-0H3D74hSxCG1CgH4LJffk4FFj17FE4ftxBNLVr/s640/dev_disk_2.png" width="640" /></a></div>
<br />
(If there are multiple partitions showing on the USB drive the file -sL command can be used to get more information if you're not sure which one to mount.)<br />
<br />
Once you've determined the USB device keep this handy for the mount command. The next few commands and outputs are the same for the unencrypted and encrypted system. <br />
<br />
In order to mount the USB drive, the system drive will need to be changed to read/write by using mount -uw:<br />
<div class="code">
mount -uw /</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixnoRd3lELfjHEbkbG_B8VMrWtWsZh69_X8HvrbHjb4tWU7mw8yvdZQqo7lU1g7IpjMH8Nqegy2ppXa3jBAPM5PyNsWXY5OvXb3Na78ijQlWQKBCrB0nz6iYYHcF4eZ9yb8teshTWpICk7/s1600/mount_uw.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="58" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixnoRd3lELfjHEbkbG_B8VMrWtWsZh69_X8HvrbHjb4tWU7mw8yvdZQqo7lU1g7IpjMH8Nqegy2ppXa3jBAPM5PyNsWXY5OvXb3Na78ijQlWQKBCrB0nz6iYYHcF4eZ9yb8teshTWpICk7/s640/mount_uw.png" width="640" /></a></div>
<br />
[EDIT 8/1/2016] **** Please read the comments. There is information on how you can mount without changing the system drive to read/write****[END EDIT]<br />
<br />
Next, a mount point will need to be created for the USB drive. For this example, the mount point will be created under /tmp/usb: <br />
<br />
<div class="code">
ls /tmp <br />
mkdir /tmp/usb</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzYO54SBx8k0vcL5IHIcr2OI5seMMIm1AudrkSnT2y86tZlHCCX_dmV8wSMsciCrgSGON4IVzwB3f_8eczGCKUa8t41HFU3bSIVvK69m36kSjsu8_jbRqCsXbIY3T_qn6ZklYq4QO4jzRX/s1600/ls_tmp_usb.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="92" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzYO54SBx8k0vcL5IHIcr2OI5seMMIm1AudrkSnT2y86tZlHCCX_dmV8wSMsciCrgSGON4IVzwB3f_8eczGCKUa8t41HFU3bSIVvK69m36kSjsu8_jbRqCsXbIY3T_qn6ZklYq4QO4jzRX/s640/ls_tmp_usb.png" width="640" /></a></div>
<br />
<br />
Now it's time to mount the USB drive.The mount command will need the partition type (FAT32 or HFS), the disk to mount, and a mount point.<br />
<br />
<br />
<b>Mount the FAT32 USB drive on the unencrypted system</b><br />
To mount the FAT32 drive on the unencrypted system the following syntax was used:<b> </b><br />
<br />
<div class="code">
mount -t msdos /dev/disk1s1 /tmp/usb</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvTNXpSB4MIVnM8gL2dUB5vQv4GfknSEznBf0PZmW1FEiIlmPmh5TUldP2xT2H-RpzGuxdLEpwxA3c0iYIhVYLMiOlAX5EkmSUmBYSGNKSYuJSGoMuQH7n4rqYRU8__qgeRRll6YrYEnhT/s1600/mount_msdos.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="136" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvTNXpSB4MIVnM8gL2dUB5vQv4GfknSEznBf0PZmW1FEiIlmPmh5TUldP2xT2H-RpzGuxdLEpwxA3c0iYIhVYLMiOlAX5EkmSUmBYSGNKSYuJSGoMuQH7n4rqYRU8__qgeRRll6YrYEnhT/s640/mount_msdos.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<b>Mount the HFS Drive on the encrypted system</b><br />
To mount the hfs drive on the encrypted system, the following syntax was used:<br />
<div class="code">
mount -t hfs /dev/disk2s2 /tmp/usb </div>
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcgbzSHGrN5Wfrxwxu8KDPpKr03LFA9GvAhyphenhyphenP1y3L5E_l1rQMKIOz-DhG9QDEJ1rrRl8lDoN2VBYAFkYUQl5lp7gLWvdBe0u9NKq3ag8qr8NVAGM8R1siC88cFVjF3PPPefdzZoN6oD-8o/s1600/mount_hfs.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="106" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcgbzSHGrN5Wfrxwxu8KDPpKr03LFA9GvAhyphenhyphenP1y3L5E_l1rQMKIOz-DhG9QDEJ1rrRl8lDoN2VBYAFkYUQl5lp7gLWvdBe0u9NKq3ag8qr8NVAGM8R1siC88cFVjF3PPPefdzZoN6oD-8o/s640/mount_hfs.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
I always create a subfolder on the USB drive to hold the image. This way I can list the contents of the mount point as a sanity check to ensure that it mounted ok:<br />
<br />
<div class="code">
ls /tmp/usb</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUGRuDrimOxGBTr0z4-lUX9xcJnekHbUnTASM0Xas3MmeSpskumK34Jh5TzeikgbSmrlNCBo1yJ41efCbjXK0eJCCN9qyALwNsJrJpHrbR3N701RnwNpIxpu2gfVJP7-5fW3UZkMYibsmE/s1600/list_tmp_usb.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="114" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUGRuDrimOxGBTr0z4-lUX9xcJnekHbUnTASM0Xas3MmeSpskumK34Jh5TzeikgbSmrlNCBo1yJ41efCbjXK0eJCCN9qyALwNsJrJpHrbR3N701RnwNpIxpu2gfVJP7-5fW3UZkMYibsmE/s640/list_tmp_usb.png" width="640" /></a></div>
<br />
Here I can see "MacEncryptedImage" and "MacImage", the folders I created on the USB drive. Everything looks good to go.<br />
<br />
<u><b>Step 4 - Create the image </b></u><br />
<br />
To create the image, the dd command can be used. For dd, I use the options recommend on the <a href="http://www.forensicswiki.org/wiki/Dd" target="_blank">Forensic Wiki page.</a> The syntax looks something like this:<br />
<br />
dd if=/dev/disk0 bs=4k conv=sync,noerror of=/tmp/usb/mac_image.dd <br />
<br />
Lets break down this command: <br />
<br />
<ul>
<li>if=/dev/disk0: this stands for input file. This will be the disk that requires imaging</li>
<li>bs=4K : this is the block size used when creating an image. The Forensic Wiki recommends 4K</li>
<li>conv=sync,noerror: if there is an error, null fill the rest of the block; do not error out</li>
</ul>
<br />
<br />
If /dev/rdisk is available this can be used instead of /dev/disk. rdisk provides access to the raw disk which is<a href="http://superuser.com/questions/631592/why-is-dev-rdisk-about-20-times-faster-than-dev-disk-in-mac-os-x" target="_blank"> supposed to be faster</a> then /dev/disk which uses buffering. <br />
<br />
<b>Unencrypted system</b><br />
For the unencrypted system the image will be of /dev/disk0 to a FAT32 USB mounted drive. Since FAT32 has a 4GB file size limit, dd will need to be piped through the split command to keep the file size under 4GB:<br />
<br />
<div class="code">
dd if=/dev/disk0 bs=4k conv=sync,noerror | split -b 2000m - /tmp/usb/Images/disk0.split.</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfXG8rPt61Vhe3GSbAdgXMJivSOuw6FrddkmjOTJRDNI_A1C2MhAsxRi8NoSO6_cdY5GgFY5W2ICorQ-lSOKqSy66I-I5AVBE6xf0a6qpQhn5JcvkJQ3ySjL5P2PpZHscjqmB4s4gSqGkQ/s1600/msdos_dd_image.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="92" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfXG8rPt61Vhe3GSbAdgXMJivSOuw6FrddkmjOTJRDNI_A1C2MhAsxRi8NoSO6_cdY5GgFY5W2ICorQ-lSOKqSy66I-I5AVBE6xf0a6qpQhn5JcvkJQ3ySjL5P2PpZHscjqmB4s4gSqGkQ/s640/msdos_dd_image.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<b>Encrypted system</b><br />
For the encrypted drive, this example will be of /dev/rdisk1<b>. </b>Since the image will be saved to an HFS USB drive there is no need to split the image:<br />
<br />
<div class="code">
dd if=/dev/rdisk1 bs=4k conv=noerror,sync of=/tmp/usb/MacEncryptedImage/Mac_rdisk1.dd </div>
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZI8IF2Qmq1p0qd5d5DmgdahI5J6f0rS0dB-Nk7SYsu9wv9cbPjhIMTnfVJ6JRzFhK-XQUrEn0iENCcMZf5sX0oR-CdU0cgSHO1gIGJRdBuNULEF9wxfMGvNmlWr_Md0wDcdk02owSrv7Z/s1600/dd_encrypted_rdisk.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="84" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZI8IF2Qmq1p0qd5d5DmgdahI5J6f0rS0dB-Nk7SYsu9wv9cbPjhIMTnfVJ6JRzFhK-XQUrEn0iENCcMZf5sX0oR-CdU0cgSHO1gIGJRdBuNULEF9wxfMGvNmlWr_Md0wDcdk02owSrv7Z/s640/dd_encrypted_rdisk.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"></td></tr>
</tbody></table>
<br />
Unfortunately, dd does not have a progress bar so patience is a virtue. Once it's complete, a message similar to below should appear:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhALS4IXCkaSwKfNWnIcXG0Xnik7c14Mo69zvKtalJMxj87rxYvRMlcB-W6Z2UMjOgUVQXc7raFroJ4tvIQqCTEX9r0zN3Y7T-spJAECSbxyLEOp1q6-4VJEr9bZ2pOVkHqGfd1GUDoFu7k/s1600/dd_image_finish.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="70" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhALS4IXCkaSwKfNWnIcXG0Xnik7c14Mo69zvKtalJMxj87rxYvRMlcB-W6Z2UMjOgUVQXc7raFroJ4tvIQqCTEX9r0zN3Y7T-spJAECSbxyLEOp1q6-4VJEr9bZ2pOVkHqGfd1GUDoFu7k/s640/dd_image_finish.png" width="640" /></a></div>
<br />
<br />
<b><u>View Image</u></b><br />
As a last step, I just wanted to show how each image looked when opened in FTK Imager.<br />
<br />
<br />
<b>Unencrypted</b><br />
The unencrypted image looks as expected, three partitions in an unencrypted state:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2W79ZRLmRVAjb7XoDFwrlZlcN9X4XT_SNFIuUchBT6XsyxYZf5r-_5cLS9n8VcjAnV3dW2elrEf492R4U7hfYZid6W9XtzsEiTNcma1u2oQgBKn519Xm8H4wohxn2-2pvet5mCKbONkpH/s1600/ftk_disk0.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="420" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2W79ZRLmRVAjb7XoDFwrlZlcN9X4XT_SNFIuUchBT6XsyxYZf5r-_5cLS9n8VcjAnV3dW2elrEf492R4U7hfYZid6W9XtzsEiTNcma1u2oQgBKn519Xm8H4wohxn2-2pvet5mCKbONkpH/s640/ftk_disk0.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<b>Encrypted</b><br />
During my testing, I imaged both /dev/rdisk0 and /dev/rdisk1. /dev/rdisk0 was the entire disk with all three partitions. Opening the rdisk0 image in FTK Imager confirms that all three partitions are present. As expected partition 2, MacOSX, is showing as an unrecognized file system because it is encrypted:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIyCsGGYJXRiNAROI_QJX9IUOcyKONSCxK5swZIDWGqiDTaX9RqO5z7KgHZ9any2W8GxL0hczY0nzLmxv2hAwx4aO7Rmq8wNqi46au6tqPWOq7Jd9yt9r9aA6D8H_5ekpaqTVTEeNoLZki/s1600/ftk_rdisk0_encrypted.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="340" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIyCsGGYJXRiNAROI_QJX9IUOcyKONSCxK5swZIDWGqiDTaX9RqO5z7KgHZ9any2W8GxL0hczY0nzLmxv2hAwx4aO7Rmq8wNqi46au6tqPWOq7Jd9yt9r9aA6D8H_5ekpaqTVTEeNoLZki/s640/ftk_rdisk0_encrypted.PNG" width="640" /></a></div>
<br />
<br />
The image of /dev/rdisk1 was an image of just the second partition, which is the MacOSX partition. Opening it up in FTK Imager confirms that /dev/rdisk1 is in a decrypted state:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgo5XdCjXIafqouAZ7TZZ-ScNQbt2ZZ0T8CbGYMCqlRoi4CvLgAdqrR82ioceK0B-a4J8c_gZe1oQWcvj-HvxCCuUMrCm1DJCOCNl6fgvVd2Lh6wfzljsjwcVGc0VsvNWe7-RQvEPUMAhAj/s1600/ftk_rdisk1_encrypted.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="410" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgo5XdCjXIafqouAZ7TZZ-ScNQbt2ZZ0T8CbGYMCqlRoi4CvLgAdqrR82ioceK0B-a4J8c_gZe1oQWcvj-HvxCCuUMrCm1DJCOCNl6fgvVd2Lh6wfzljsjwcVGc0VsvNWe7-RQvEPUMAhAj/s640/ftk_rdisk1_encrypted.PNG" width="640" /></a></div>
<br />
<br />
So, in summary, here are the steps and commands covered above:<br />
<br />
<ul>
<li>Use Command-S to boot into single user mode</li>
<li>Use <i>ls /dev/disk* </i>to determine the disk(s) to image</li>
<li>Plug in the USB Drive</li>
<li>Use <i>ls /dev/disk*</i> to determine USB drive device</li>
<li>Use <i>mount -uw /</i> to change internal drive to read/write</li>
<li>Use <i>mkdir /tmp/USB</i> to create a mount point</li>
<li>Use mount to mount the USB Drive</li>
<ul>
<li>mount -t msdos /dev/disk1s1 /tmp/usb (for FAT32)</li>
<li>mount -t hfs /dev/disk2s2 /tmp/usb (for HFS)</li>
</ul>
<li>Create disk image using dd </li>
<ul>
<li>dd if=/dev/disk0 bs=4k conv=sync,noerror | split -b 2000m - /tmp/usb/disk0.split. (FAT32 USB)</li>
<li>dd if=/dev/rdisk0 bs=4k conv=noerror,sync of=/tmp/usb/rdisk0.dd (HFS USB)</li>
</ul>
</ul>
<br />
While these steps worked on my test Mac, examiners should always test and research the model they are encountering. I was limited to one test system, one hard drive and FileVault2 encryption. I also recommend trying this on a test Mac before running these steps on actual evidence. Single user-mode logs in as root, and this can be very dangerous. Remember - Trust but Verify! :)<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com4tag:blogger.com,1999:blog-2456910508235577340.post-69632309777044807012016-06-16T10:40:00.001-07:002016-06-16T10:40:23.805-07:00How to image a Mac with Live Linux bootable USB<div class="separator" style="clear: both; text-align: center;">
</div>
One thing I've learned when it's comes to imaging Macs is it's good to have options. When encountering Macs, its seems like there is always a challenge. No firewire ports for target disk mode, no easy way to remove the hard drive, or if the the hard drive is removed, you don't have the specific adapter needed to connect your write blocker to the drive... and of course, encryption. I am planning on doing several blog posts about different ways to image a Mac. Depending on the situation, some may work, some may not, but I just wanted to throw some options out in the Google soup mix.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVh8sMltbP8TWjSAuaYR0bgA0Wr9kRpX6l3GVRaAXXSIbilofkhM3hbiFC866diir1V0UjtCxFoFGfU4TEhlubcNI6-A4t4F8Nv5RhsCxfj2UnPmRqyEUHE7zMJHRYKqVOCMnc_YJ6kuSJ/s1600/boot_mac_with_linux.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="254" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVh8sMltbP8TWjSAuaYR0bgA0Wr9kRpX6l3GVRaAXXSIbilofkhM3hbiFC866diir1V0UjtCxFoFGfU4TEhlubcNI6-A4t4F8Nv5RhsCxfj2UnPmRqyEUHE7zMJHRYKqVOCMnc_YJ6kuSJ/s320/boot_mac_with_linux.png" width="320" /></a></div>
<br />
The first option I am going to go walk through is imaging a Mac with a Live Linux bootable USB. Many times cracking open something like a MacBook Air to grab a hard drive requires special tools and adapters which may not be readily available. If the Mac is already powered off, booting the Mac with a live Linux distro may be a good option. Once booted into Linux, an imaging tool with a GUI, like Guymager, can be used to create an image in E01 or dd format.<br />
<br />
For this post, I have selected the <a href="http://www.caine-live.net/" target="_blank">CAINE</a> distro. CAINE stands for Computer Aided Investigated Environment. This distro was
made specifically for computer forensics. Upon boot, CAINE "blocks all
the block devices (e.g. /dev/sda), in Read-Only mode." The examiner must take active steps, which includes nice
big warnings, to turn off this feature.<br />
<br />
While I did get Kali to work, it did not seem to offer the extra protection that CAINE did to keep the examiner for inadvertently mounting the wrong drive. If you are interested in making a Kali bootable USB drive for the Mac, I have included some brief instructions at the bottom of the post.<br />
<br />
This method was tested with CAINE 7.0, Rufus 2.9, and a MacBook Air Early 2015 model<span style="color: red;"> </span><br />
<br />
<b>Create the Bootable USB</b> <br />
<br />
The first step is to create a bootable USB drive on a Windows machine. Download the<a href="http://www.caine-live.net/page5/page5.html" target="_blank"> CAINE iso</a> and <a href="https://rufus.akeo.ie/" target="_blank">Rufus</a>. Rufus is the Windows program that will create a bootable USB drive from the iso. Simply launch Rufus and select the CAINE iso as well as a blank USB drive bigger than 4GB. (NOTE - I tried various other tools to create the bootable USB drive, and not all of them worked when it came time to boot the Mac. Thanks to @APFMarc for the tip on Rufus). Below is a screen shot with the settings I used:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiogYwq1a2B3-MlIuZ_NZsnuvRZH7BJLb6KZNeJKRlJLs9E5mSZyoSOZfpUsu4cWURU_z3oDFYSb7_Wbhkod0samy7UGoiSHXbP_Y4AGE9siIYhA6h1kV6pLkq_ubyLG4WR66oYIIHNemsn/s1600/rufus.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiogYwq1a2B3-MlIuZ_NZsnuvRZH7BJLb6KZNeJKRlJLs9E5mSZyoSOZfpUsu4cWURU_z3oDFYSb7_Wbhkod0samy7UGoiSHXbP_Y4AGE9siIYhA6h1kV6pLkq_ubyLG4WR66oYIIHNemsn/s1600/rufus.PNG" /></a></div>
<br />
There was a pop up dialog box when I clicked start asking me to choose to write in ISO image mode or DD Image mode. I used the default, which was ISO mode. <br />
<br />
<b>Boot into Linux</b><br />
<br />
Once completed, this USB drive can be used to boot the Mac. In order to boot a Mac from a USB device, it must be put into <a href="https://www.tekrevue.com/tip/mac-startup-options/" target="_blank">Startup Mode</a>. This is done by holding down the Alt/Option button when the systems boots. Once in Startup Mode, the boot device can be selected. The CAINE USB should show up as the EFI Boot choice:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHTvw_Qs9l_eXDRtpZynbg4JlMs2QQoeK38vRq56brLfIgnYVLMWCAJc8XYrE97xXe9U_cwXSNzhnN8dq7p3DfoiI-4z5i0KIxyU-g3kwWWDFvj2Rg3hpi0qoVkc3p7E6HvVpKL2QuSDox/s1600/efi-mode.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHTvw_Qs9l_eXDRtpZynbg4JlMs2QQoeK38vRq56brLfIgnYVLMWCAJc8XYrE97xXe9U_cwXSNzhnN8dq7p3DfoiI-4z5i0KIxyU-g3kwWWDFvj2Rg3hpi0qoVkc3p7E6HvVpKL2QuSDox/s320/efi-mode.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1DyctRgERW4X8_4oEEwWey0oLM9VHTWcZ_EoHMSWeE3o1nzTTH1y0M0mSnNQz6_uYeNx_8prFADm1OLvuxTrkWQC5_l8BaWQSom_HmEdxBATZDALC7UjJ3SofUVGq9o5sCb_lXWLOzVU0/s1600/efi-mode.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><br /></a></div>
<br />
After CAINE boots, choose the "Boot Live system". If all goes well, the following desktop should appear: <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIBbWNS0y2sZ2oC5GXCkQZMFhfQb994TFwNP3mtRBLrYpIuz4dECrxu_OG15APhtxAi6wFY3YqAGNBibGcsmlGKTA8hvxFh0wlOt-kPPAVppgZLGQyTiZ2AkSpODZ7qizV_YZ7fE7k0sLr/s1600/CAINE_Desktop.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIBbWNS0y2sZ2oC5GXCkQZMFhfQb994TFwNP3mtRBLrYpIuz4dECrxu_OG15APhtxAi6wFY3YqAGNBibGcsmlGKTA8hvxFh0wlOt-kPPAVppgZLGQyTiZ2AkSpODZ7qizV_YZ7fE7k0sLr/s640/CAINE_Desktop.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<br />
CAINE has a utility called Mounter, which is located in the task bar. It's the tiny icon circled above. Double clicking this icon brings up a dialog box that shows which block devices are currently mounted:<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi37d_o9XrsxtXMpA66RCEJySRaujjcuXN5BI0SR-3z7PvFFAA5B1LEmbCDUj-5GEn8o5jsoH_KNFOCq_7A30C4h_m3EJZSMhZRSVtgv0ZvBr3r6FTXJZJsLBVhFinCOEMJn8bD-vntRY3W/s1600/mounter.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="316" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi37d_o9XrsxtXMpA66RCEJySRaujjcuXN5BI0SR-3z7PvFFAA5B1LEmbCDUj-5GEn8o5jsoH_KNFOCq_7A30C4h_m3EJZSMhZRSVtgv0ZvBr3r6FTXJZJsLBVhFinCOEMJn8bD-vntRY3W/s640/mounter.png" width="640" /></a></div>
<br />
<b> </b>As demonstrated in the screen shot above, the only device that is currently mounted is the USB containing the CAINE distro (Shown as CAINLIVE). Running the df command also confirmed this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWX9vrQ03vHyyJRuyNgzkfKyxMPsiJJ3ZPunoNoyt6ixOej5pxA8pn6_7rKv2on06vJ500xXhEmwBBI_9Sm5Dy15cq-mOxD57RFQ29zIXMboWiA2UAG-pFOWBmtd_MWrhFTQf9q5pRYjkM/s1600/CAINE_df_command.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWX9vrQ03vHyyJRuyNgzkfKyxMPsiJJ3ZPunoNoyt6ixOej5pxA8pn6_7rKv2on06vJ500xXhEmwBBI_9Sm5Dy15cq-mOxD57RFQ29zIXMboWiA2UAG-pFOWBmtd_MWrhFTQf9q5pRYjkM/s640/CAINE_df_command.png" width="640" /></a></div>
<br />
<br />
The reason I like CAINE is that it does not let the examiner inadvertently mount a drive by accidentally clicking on something. For example, when I double clicked the Macintosh HD it gave me an error: <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLFjZcX92X14jfAHwiLQwAV5EVtPVtX7j0ogXB8-oCsqoYUldoXjRa5ghyiQVfzzsDPScBn5r4XizgMFm9KCarWpIlPV-qsScinD9TxyGT1lCESlOg7T744TG1-sY8FjLJKcbzCAvdW9jz/s1600/unable+to+mount.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="494" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLFjZcX92X14jfAHwiLQwAV5EVtPVtX7j0ogXB8-oCsqoYUldoXjRa5ghyiQVfzzsDPScBn5r4XizgMFm9KCarWpIlPV-qsScinD9TxyGT1lCESlOg7T744TG1-sY8FjLJKcbzCAvdW9jz/s640/unable+to+mount.png" width="640" /></a></div>
<br />
<br />
<br />
From the CAINE website documentation:<b> </b><br />
<blockquote class="tr_bq">
This new write-blocking method assures all disks are really
preserved from accidentally writing operations, because they are locked
in Read-Only mode.<br />
If you need to write a disk, you can unlock it with
BlockOn/Off or using "Mounter" changing the policy in writable mode. </blockquote>
I personally prefer this extra layer of protection.<br />
<br />
<b>Mount the USB drive that will hold the image</b><br />
<br />
Next, an external USB drive is needed to dump the image on. This external device needs to be mounted writable so the image can be placed on it. To do this, Read only mode needs to be turned off for any newly attached devices by using the Mounter program. Right clicking the Mounter icon in the task bar brings up the following dialog box:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjrHwICsyYdTbTJvpVIsqvWrvNUGl_w2kQJKg1hZN4B8NGsnlGh2PtIV9B2UahtAYq991pTqnh6bQaYR7hpAI0mMVcd6_AJ1XOHi9WDgTHbJ4gJj5LzwbcvVrYjb3LXCpUBWfvhcUdLrdy/s1600/make+writeable.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="160" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjrHwICsyYdTbTJvpVIsqvWrvNUGl_w2kQJKg1hZN4B8NGsnlGh2PtIV9B2UahtAYq991pTqnh6bQaYR7hpAI0mMVcd6_AJ1XOHi9WDgTHbJ4gJj5LzwbcvVrYjb3LXCpUBWfvhcUdLrdy/s320/make+writeable.png" width="320" /></a></div>
<br />
<br />
I know this looks scary, all in red and what not, but clicking it brings up <b><i>another</i></b> dialog box confirming that this action will only make <i><b>newly</b></i> mounted devices writable, which is what is needed so the image can be dumped to the external drive:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiq5mBWwcRBRZ3cvwPZMNJI0nDhNes5iMP4Gt-AfmhqY2Y4L7Da15jQ7m2VJ199NNZc7mdn5-ZM9mQQJS1pLbt_nJ7zim84jiGkvUEoNmmV24V5lBbuB3LzfcoutPflzUV8eraB1_hdkbym/s1600/mounter-warning.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiq5mBWwcRBRZ3cvwPZMNJI0nDhNes5iMP4Gt-AfmhqY2Y4L7Da15jQ7m2VJ199NNZc7mdn5-ZM9mQQJS1pLbt_nJ7zim84jiGkvUEoNmmV24V5lBbuB3LzfcoutPflzUV8eraB1_hdkbym/s320/mounter-warning.png" width="320" /></a></div>
<br />
<br />
After selecting Yes, a brief confirmation message pops up and the icon in the tray also turns red indicating the current status:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7GJ2eDWtVz5A1v76WCVGrIaQbQhW2ViMu0hoMkq77O_R-NU4sQ7br4-xB3fMwE6VY32-O-r47kXqn00e4_c3ef_liVZQ8o1clQ8WdFFDksHQGR2XENhQvx2BvvKG1guDycXiTYaVSmgcs/s1600/write_policy_mounter.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="90" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7GJ2eDWtVz5A1v76WCVGrIaQbQhW2ViMu0hoMkq77O_R-NU4sQ7br4-xB3fMwE6VY32-O-r47kXqn00e4_c3ef_liVZQ8o1clQ8WdFFDksHQGR2XENhQvx2BvvKG1guDycXiTYaVSmgcs/s400/write_policy_mounter.png" width="400" /></a></div>
<br />
The next step is to plug in the USB drive that will contain the image. I named my external USB drive "Images" and formatted it with NTFS on a Windows system before beginning this process (FAT32 will work as well, just be aware of the 4GB file limit). Once the drive is plugged in, it can be mounted by opening up the "Caine's Home" folder on the desktop and double clicking the drive. This will mount the drive to /media/CAINE/YourDriveName:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYSvpLlPief0jR7n9OP4Q8dYgr83J7pQrLQpXc0SGdN-MCFLikRrhoKKkHRrhftHIK3JQluLlGkrnNaAS1aoQ-sFqwgjrnHW0Pqv9pJmIO8WEuzo7g-exX8NxxwYftEzKGb6w5BLe6kgcC/s1600/mount_external_drive.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="492" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgYSvpLlPief0jR7n9OP4Q8dYgr83J7pQrLQpXc0SGdN-MCFLikRrhoKKkHRrhftHIK3JQluLlGkrnNaAS1aoQ-sFqwgjrnHW0Pqv9pJmIO8WEuzo7g-exX8NxxwYftEzKGb6w5BLe6kgcC/s640/mount_external_drive.png" width="640" /></a></div>
<br />
Now the drive can be accessed to create folders, dump the image to etc. Note - if I try and do the same with the other devices on the host drive (e.g. BOOTCAMP and Macintosh HD), it will give me an error, thereby preventing me from accidentally mounting them.<br />
<br />
<b>Use Guymager to create the image</b><br />
<br />
Now that the external USB drive is mounted, Guymager can be started to to create the image. Guymager is found on the Desktop, or under Menu>Forensic Tools>Guymager.<br />
<br />
Once launched, select the device that needs to be imaged by right clicking it. In this example, the drive I want is the "ATA Apple SSD SM0256G":<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtLQg71VUF8eNvunJ5l-ghYJGCtvXsCHrGmP6s674mKojMPivrGTIDRWmHdbT2UqOGCnavCTZdDdKV9rYFY1XZuPo0vhA1E5j6OVWfKRliQUX8Y-2N1rkax-OkruB0Y6qSzGYbKK9uByAl/s1600/guymager-aquire.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="384" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtLQg71VUF8eNvunJ5l-ghYJGCtvXsCHrGmP6s674mKojMPivrGTIDRWmHdbT2UqOGCnavCTZdDdKV9rYFY1XZuPo0vhA1E5j6OVWfKRliQUX8Y-2N1rkax-OkruB0Y6qSzGYbKK9uByAl/s640/guymager-aquire.png" width="640" /></a></div>
<br />
<br />
The next step is to fill out all the requisite image data:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGN1EM76_2UllZsye75XCKyXJGmHYCEYp4nqFc8u8V90nZUtNquWj1dHEEA0VFJmoiuJiMWmkRW-_6biwZzlhhHv2GWDE6IhE6tWUwapCTvkNPRk6M7ROM1Nvc8U88CzHOcny_J_OY_O82/s1600/aquire-image.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="524" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGN1EM76_2UllZsye75XCKyXJGmHYCEYp4nqFc8u8V90nZUtNquWj1dHEEA0VFJmoiuJiMWmkRW-_6biwZzlhhHv2GWDE6IhE6tWUwapCTvkNPRk6M7ROM1Nvc8U88CzHOcny_J_OY_O82/s640/aquire-image.png" width="640" /></a></div>
<br />
<br />
Once started, the previous table will show a status on the imaging process. When the image is complete Guymager will create a log file in the same directory as the image. An interesting tidbit - a while back, Eric Zimmerman did some <a href="https://docs.google.com/spreadsheets/d/1wXX5zYql7KIPgrsDdt6S5bTuGt_WRjWaBde1D0fhG5k/edit?type=view&gid=0&f=true&sortcolid=11&sortasc=true&rowsperpage=250#gid=0" target="_blank">testing</a> on various imaging tools, and Guymager was one of the fastest :)<br />
<br />
<br />
<b>Kali live Linux bootable USB for Mac </b><br />
<br />
So far, I've only found one method that works consistently to boot into Kali Linux on a Mac (at least on my test Mac). That method is to use the <a href="https://sevenbits.github.io/Mac-Linux-USB-Loader/" target="_blank">Mac Linux USB Loader</a> on a Mac to create the bootable USB.<br />
<br />
There is a video here that has step by step instructions for the Mac Linux USB Loader, but it's pretty straight forward to use. The basic steps are
1) Download <a href="https://www.kali.org/" target="_blank">Kali Linux;</a> 2) Using Disk Utilities on Mac to format a USB drive
with Fat32 and MBR; 3) Run Mac Linux USB Loader and select the Kali Iso;
4) Choose Kali from the distro type<br />
<br />
You can now boot into Kali and use Guymager
on a Mac using the same steps I detailed in the sections above. One very important thing to note - using this method will automatically boot you into the Kali Live environment and you will not be given the choice for the Kali Linux Forensics Mode.<br />
<br />
In my limited testing it does not appear to mount the host drive, or make any changes to the drive. It also does not have the additional steps and warnings when it comes to inadvertently mounting drives that CAINE does. The Live version will also auto-mount plugged in USB devices. Proceed at your own risk, and as they state on the Kali website:<br />
<blockquote class="tr_bq">
If you plan on using Kali for real world forensics of any type, we
recommend that you don’t just take our word for any of this. All
forensic tools should <i>always</i> be validated to ensure that you know how they will behave in any circumstance in which you are going to be using them</blockquote>
Echoing these same sediments, although I have walked though a method of imaging a Mac from a live Linux distro, please test and validate before using either of these methods in the the real world.<br />
<br />
Happy Mac-ing! <br />
<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com4tag:blogger.com,1999:blog-2456910508235577340.post-91472586744733581552016-05-16T08:40:00.000-07:002016-05-16T08:40:17.919-07:00QuickLook Python Parser - all your BLOBs belong to usI've always mentioned in my presentations and blog posts that if anyone needs any help parsing an artifact, to hit me up - I love working on these types of projects in my spare time. Matthew Feilen (@mattevps) did just that. The artifact in need of parsing was the index.sqlite file which is part of the<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIMMZwu2FdX-JgWsXI4SaNH7-pKZhMVn7uPrXOq0T8bTWeyu-kHXg2okK7MsrcjbFKe4u2zhT1gGHtJsxr86Oc-NIV2M_XANTQ3wPZyuKO9u8YV3sUK_D0GzGno_rF1V56hV9MCAvYkhdw/s1600/Blob.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIMMZwu2FdX-JgWsXI4SaNH7-pKZhMVn7uPrXOq0T8bTWeyu-kHXg2okK7MsrcjbFKe4u2zhT1gGHtJsxr86Oc-NIV2M_XANTQ3wPZyuKO9u8YV3sUK_D0GzGno_rF1V56hV9MCAvYkhdw/s320/Blob.jpg" width="265" /></a></div>
OS X QuickLook feature. While an SQL query can pull most of the data, there is a plist file stored as a BLOB (Binary Large Object) that needs to be parsed. This BLOB has additional data that can be useful to an examiner. Read on for more details.<br />
<br />
<b>QuickLook Background</b> <br />
<br />
The QuickLook database stores information about thumbnails that have been generated on a Mac.<br />
This information includes things like the file path to the original file, a hit count, the last date and time the thumbnail was accessed, the original file size and last modified date of the original file.<br />
<br />
The cool thing is that this database can contain entries after a file has been deleted as well as entries for externally mounted volumes, like thumb drives. This database can also persist after a user account has been deleted since it's not located in a user directory. Sara Newcomer wrote an excellent <a href="http://iacis.org/iis/2014/10_iis_2014_421-430.pdf" target="_blank">white paper</a> that details this artifact. I suggest reading her white paper for the finer points since my focus will be mainly on parsing the data out.<br />
<br />
There is an index.sqlite file for each user on the system. These files are
located under
/private/var/folders/<random>/<random>/C/com.apple.QuickLook.thumbnailcache. The <random><random> will be different for each user. Since this database is not stored under a user's folder, you will need to tie the index.sqlite to a user by checking the permissions on the file. If you're on a live system, it's pretty easy to do with the ls -l command. However, if you have an image it may be a little more involved. One way I found to do this is to check the owner properties on the file, than cross reference this to the user's plist file. In the example below, I've used FTK Imager to view the UID of the index.sqlite file, which is 501:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimsg7S-yAjL5HX1a1siTs5LIkbTJR7nhQE0tv646FuEQP5aUkFwsM-L2RxoLYwn7lZA6-ktZb_QxqULF8JRH4xwoEcZ9oskuV71SLsXYoCwVZg8qQLjIZYYMh6yGvxYOra1K2G1RJudrp7/s1600/mac-UID.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimsg7S-yAjL5HX1a1siTs5LIkbTJR7nhQE0tv646FuEQP5aUkFwsM-L2RxoLYwn7lZA6-ktZb_QxqULF8JRH4xwoEcZ9oskuV71SLsXYoCwVZg8qQLjIZYYMh6yGvxYOra1K2G1RJudrp7/s320/mac-UID.png" width="256" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">UID of the index.sqlite file</td></tr>
</tbody></table>
Next, I exported out the user's plist file located under /private/var/db/dslocal/node/Default/users and used <a href="http://www.icopybot.com/plist-editor.htm" target="_blank">plist editor</a> to locate the UID for that user:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmW6px0o-08YgJ4405nNsdO-R4ih2BYcNamO_AjiJb3bPfyAEfg7OGF5hE6jXmOuoU6Y6xU9yIPWDlP5DEBv4bRHuf4qfMRks38HUAV-HGKAwSVkt305GcwiB7Mj8B5ST6O4UbUOk6ttRl/s1600/maridegrazia-plist.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjmW6px0o-08YgJ4405nNsdO-R4ih2BYcNamO_AjiJb3bPfyAEfg7OGF5hE6jXmOuoU6Y6xU9yIPWDlP5DEBv4bRHuf4qfMRks38HUAV-HGKAwSVkt305GcwiB7Mj8B5ST6O4UbUOk6ttRl/s1600/maridegrazia-plist.PNG" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">UID in user plist file</td></tr>
</tbody></table>
<br />
<br />
<b>Getting the Data Out</b><br />
<br />
There are two tables of interest in the QuickLook index.sqlite file, the "files" table and the "thumbnails" table. Most of the information contained in these two tables can be pulled with an SQL query. In fact, a <a href="http://www.easymetadata.com/2015/01/sqlite-analysing-the-quicklook-database-in-macos/" target="_blank">blog post</a> by a "Dave" details how to write an SQL query to join these two tables and decode the Mac absolute timestamps. However, the "files" table contains a field named "version" that contains a BLOB. This BLOB, aka binary data, is a plist file:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEoCApT6qz2nqWUJa3f9BZhx87lz0wLn1gRHyNw5PeHmNnOvXD2QCj0GtWSvbSoEM0bkzMH4lBZJy4iKLbzH2C9Mcxn_hLwFH0xF-dyfFqltaWuTPbXmc5klKoYWuH653wwAhtbPVORUuf/s1600/blob.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEoCApT6qz2nqWUJa3f9BZhx87lz0wLn1gRHyNw5PeHmNnOvXD2QCj0GtWSvbSoEM0bkzMH4lBZJy4iKLbzH2C9Mcxn_hLwFH0xF-dyfFqltaWuTPbXmc5klKoYWuH653wwAhtbPVORUuf/s1600/blob.png" /></a></div>
<br />
This embedded plist file contains the last modified date of original file in Mac absolute time, the original file size, and the plugin used to generate the thumbnail:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuQHt6TtlZCKovSmHUK-idkHz59FEn6nHi7KC1cBrxt0pKTND8R3JIeTA6RR5M-fFubsUQV00NKaQqwdxQR8AH1tEtPrUDNijjdPsAJBFXw5FgQq3SM2byE7sHFWaTqstnQVUyzgNBvus6/s1600/flower_plist.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhuQHt6TtlZCKovSmHUK-idkHz59FEn6nHi7KC1cBrxt0pKTND8R3JIeTA6RR5M-fFubsUQV00NKaQqwdxQR8AH1tEtPrUDNijjdPsAJBFXw5FgQq3SM2byE7sHFWaTqstnQVUyzgNBvus6/s1600/flower_plist.PNG" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<br />
While the SQL statement works on most of the data, an additional step is needed to parse the embedded plist file from the BLOB. The data in this plist file could be helpful to an examiner, especially if it contains information about a file that no longer exists on a system.<br />
<br />
<b>Python to the Rescue </b><br />
<br />
There is a python library called <a href="https://pypi.python.org/pypi/biplist/1.0.0" target="_blank">biplist</a> that does a great job of parsing binary plists. Using this library and the SQL syntax provided in <a href="http://www.easymetadata.com/2015/01/sqlite-analysing-the-quicklook-database-in-macos/" target="_blank">the blog pos</a>t by Dave, I was able to create a python parser in pretty short order for this artifact.<br />
<br />
<br />
The syntax is pretty simple, just point it to the index.sqlite file:<br />
<br />
<div class="code">
quicklook.py -f index.sqlite >> output.tsv</div>
<br />
If you don't already have biplist installed, it can be installed by running:<br />
<br />
<div class="code">
sudo easy_install biplist</div>
<br />
I've also included a compiled executable for Windows on my <a href="https://github.com/mdegrazia/OSX-QuickLook-Parser" target="_blank">github</a><br />
<br />
The output looks like this with the parsed BLOB information in the Version column. As you can see in the example below, there is information for files on my system, as well as for files on a Lexar thumbdrive:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3xB_IAa7BQNEn9l22gzKV4gAX-5NIp1Hy0lRDNjZQh4pzFXj3dxVJcDNAfqIV1yY4Hdb-wM8m3Wu7yrcr3mBUMCKj9sKd3pw0Oc5OB8bsJAoWoySiZLX3nyATAYEZjgJ7MRXxGQG2yDZ5/s1600/output.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3xB_IAa7BQNEn9l22gzKV4gAX-5NIp1Hy0lRDNjZQh4pzFXj3dxVJcDNAfqIV1yY4Hdb-wM8m3Wu7yrcr3mBUMCKj9sKd3pw0Oc5OB8bsJAoWoySiZLX3nyATAYEZjgJ7MRXxGQG2yDZ5/s1600/output.PNG" /></a></div>
<br />
A huge thanks to Matt for contacting me about this artifact and supplying me with several test index.sqlite files. The Quicklook index.sqlite parser (both python and executable) can be <a href="https://github.com/mdegrazia/OSX-QuickLook-Parser" target="_blank">downloaded </a>from my github.<br />
<br />
<br />
Resources:<br />
<br />
Sara Newcomer's detailed article on the artifact: <a href="http://iacis.org/iis/2014/10_iis_2014_421-430.pdf">http://iacis.org/iis/2014/10_iis_2014_421-430.pdf</a><br />
Dave's blog post covering the SQL query: <a href="http://www.easymetadata.com/2015/01/sqlite-analysing-the-quicklook-database-in-macos/">http://www.easymetadata.com/2015/01/sqlite-analysing-the-quicklook-database-in-macos/</a><br />
<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com5tag:blogger.com,1999:blog-2456910508235577340.post-91847520457050900072016-02-22T06:00:00.000-07:002016-02-22T06:00:17.930-07:00More on Trust Records, Macros and Security, Oh My!<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
There is a registry key that keeps track of which documents a user has enabled editing and macros for from untrusted locations. This happens when the user clicks the "Enable Editing" button on the Microsoft Office Protect View warning:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPfUsEeo2uF6_IC1-cKH3cnRXWGOSRoF6ADSkg1owxlApFUtwIrincl_Hsap4DiohAn1NjqIqsIKjnTqLkm4QTbQqoH0LdyoQKCa8tHtwXUDYryA7CGcB8dismV5mp9M6CUkTT_-EjdFQS/s1600/protected-view.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="140" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPfUsEeo2uF6_IC1-cKH3cnRXWGOSRoF6ADSkg1owxlApFUtwIrincl_Hsap4DiohAn1NjqIqsIKjnTqLkm4QTbQqoH0LdyoQKCa8tHtwXUDYryA7CGcB8dismV5mp9M6CUkTT_-EjdFQS/s640/protected-view.PNG" width="640" /></a></div>
<br />
<br />
These can include documents that are downloaded from the Internet, or sent via email. This registry key is affectionately known as "Trust Records". When a user clicks this warning, an entry is made under HKCU\Software\Microsoft\Office\15\Word\Security\Trusted Documents\TrustRecords that contains the file path to the document (the version number may vary - I've tested 14 and 15).<br />
<br />
This is by no means a new artifact. There are several blog posts that discuss this artifact, including one by <a href="http://forensicartifacts.com/2012/07/ntuser-trust-records/" target="_blank">Andrew Case</a> and <a href="http://windowsir.blogspot.com/2012/07/links-and-updates.html" target="_blank">Harlan Carvey</a> - however, I believe I may have some new light to shed on this artifact. Well, I couldn't find the information by using Google, so it's new to me.<br />
<br />
What I found was that an entry can exist under this key, <i><b>but that does not necessarily mean that macros were enabled.</b> </i>In order to determine if macros were enabled, a flag/value needs to be checked in the binary data. Additionally, the Trust Center macro settings may need to be checked as well. The user can turn off this security prompt in the Trust Center and trust<b> all documents </b>by default. If this happens, no entry will be made under the Trust Records because all documents are trusted.<br />
<br />
Why all the fuss over macros? Who uses them anyways??? Take for example the latest ransomware variant, <a href="https://nakedsecurity.sophos.com/2016/02/17/locky-ransomware-what-you-need-to-know/" target="_blank">Locky</a>. Locky utilizes macros in a Word document to pull down it's payload. After a company get hits with something like this, they may want to know "How did this happen?" and "How can we prevent it in the future?". <br />
<br />
The Trusted Records registry key can help answer these questions. Did the user take affirmative steps by enabling editing in the document? Did they take another step and enable the macros? If so, the company may need to spend more time training employees on better security practices. Was the system setup to trust all documents by default? If so, they may need to reconfigure their GPO.<br />
<br />
The Trusted Records key can also contain references to artifacts that may no longer exist on the system, add context to your timeline, and demonstrates that a user explicitly interacted with the file.<b> </b><br />
<br />
<b>Trusted Records Registry Key </b><br />
<br />
In Word 2010 (v.14) and 2013 (v.15) there are actually two yellow banners presented to the user when macros are in a Word document. The first asks the user to "Enable Editing":<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7h6mfJT4i6L-eQnXjxUV_M7UnPcyvGuY6GiANWvOVBnbmX4sDeN72nWsltFLeB0-UAY1WW07eAs7m5PwO50syYkjjJE9lbzdh_QyrxkF0Q1bdiOeu6xu_Fp3AEj5xBcORsyrbVCxLh9nF/s1600/document-enable-editing.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7h6mfJT4i6L-eQnXjxUV_M7UnPcyvGuY6GiANWvOVBnbmX4sDeN72nWsltFLeB0-UAY1WW07eAs7m5PwO50syYkjjJE9lbzdh_QyrxkF0Q1bdiOeu6xu_Fp3AEj5xBcORsyrbVCxLh9nF/s1600/document-enable-editing.PNG" /></a></div>
<br />
After this button is clicked, an entry is created in the registry with the document name, path and time stamp. According to some <a href="http://windowsir.blogspot.com/search?q=trust+record" target="_blank">testing that Harlan did</a> (and the testing I did confirmed this as well), the time stamp is the create date of the document, NOT the time the user enabled editing:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkfFK_s2-XEXdT6RVoqg3hJgIbqPZ4y5czK1wBnp2aNfcTJ1FzO334zX6fr_GUfr9y2uhWpIypJf1RdXMMpzsam5EX-pbHksp1XBy9hfbEa3uxnjZQyXCkfPzMIVib7yfC748co3FQrqiO/s1600/reg-trustrecord-enable-editing.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkfFK_s2-XEXdT6RVoqg3hJgIbqPZ4y5czK1wBnp2aNfcTJ1FzO334zX6fr_GUfr9y2uhWpIypJf1RdXMMpzsam5EX-pbHksp1XBy9hfbEa3uxnjZQyXCkfPzMIVib7yfC748co3FQrqiO/s1600/reg-trustrecord-enable-editing.PNG" /></a></div>
<br />
The output from the Regripper plugin trustrecords is displayed below:<br />
<br />
trustrecords v.20120716<br />
Word<br />
LastPurgeTime = Thu Oct 8 20:38:08 1970<br />
Sat Feb 20 14:25:53 2016 -> %USERPROFILE%/Downloads/test-document.doc<br />
<br />
At this point in time, I have NOT clicked the second button to enable macros, yet an entry was made under this key.<br />
<br />
After I enable editing, a second banner pops up asking me if I would like to "Enable Content", which will enable the macros:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4LXVllJH1SPSVSzmPTl8iQmgfJNajDMYz7KWdq5bGMysY9N83950k5oRL3JeOS6QinrkM-6xsMz_KfVEiRZFKi8sIaElOxOSHNdXIvwfAppCQcyX3ilOe7uSP-b9RBme-ZkH40Jw6dJkM/s1600/document-enable-macros.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4LXVllJH1SPSVSzmPTl8iQmgfJNajDMYz7KWdq5bGMysY9N83950k5oRL3JeOS6QinrkM-6xsMz_KfVEiRZFKi8sIaElOxOSHNdXIvwfAppCQcyX3ilOe7uSP-b9RBme-ZkH40Jw6dJkM/s1600/document-enable-macros.PNG" /></a></div>
<br />
After I clicked this (based on my testing) the last for bytes in the binary data changes to FF FF FF 7F:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMoSeGL8Myv1n3u_Tte98J-QH1kgNea_U6dD2E_I4DoQb7nMCG3XxXCwEdv2r8uTVqelfAE47G34vGc5CUgUe0Dmb31-dYk54RGK24lpgbjmyWCrGmzWiJ49WIa1UMwF5pEaTwY5XMQPrj/s1600/reg-trustrecord-enable-macro.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMoSeGL8Myv1n3u_Tte98J-QH1kgNea_U6dD2E_I4DoQb7nMCG3XxXCwEdv2r8uTVqelfAE47G34vGc5CUgUe0Dmb31-dYk54RGK24lpgbjmyWCrGmzWiJ49WIa1UMwF5pEaTwY5XMQPrj/s1600/reg-trustrecord-enable-macro.PNG" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
This means in order to determine if the user enable macros, these last four bytes needs to be checked.<br />
<br />
<b>Security Registry Key</b><br />
<br />
The user can completely bypass this yellow banner by disabling the macro notifications. This means that an entry will not be recorded under the Trusted Document key even though the user ran a malicious document containing macros downloaded from the Internet. These setting are controlled by the Trust Center under Options > Trust Center > Macro Settings. There are four security levels to choose from:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEih58KrO9DnXYJUkna9H2hSQ0LhYNs8vV2Vg8_BztVNc1Ezxq7TgWqVBX0QF-qKbS0KnUDZuBIeVXeT5FGC_VVPdCD-swzABF-Mw38STpfXZ3s7gvn4MQoi8164wC5StN7VrL8vPvsFOMCE/s1600/trust-center-settings.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEih58KrO9DnXYJUkna9H2hSQ0LhYNs8vV2Vg8_BztVNc1Ezxq7TgWqVBX0QF-qKbS0KnUDZuBIeVXeT5FGC_VVPdCD-swzABF-Mw38STpfXZ3s7gvn4MQoi8164wC5StN7VrL8vPvsFOMCE/s1600/trust-center-settings.PNG" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
These setting are stored under the registry key HKCU\Software\Microsoft\Office\15.0\Word\Security\. Based on my testing, if the user has not altered the default settings, this key does not contain the value "VBAWarnings". However, if changes are made to the default settings, an entry for VBAWarrnings will appear, and will have a DWORD value:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWPSoy7g1EFJvLupKgznZDoJGXnCIakhBRekoqX2RMexRympf6ODdgTlSDsPE0PnHFEn2KrhZWrWnqSnrDhxG-aGmOYayDxbIP85PUzVe3UotTFQPc4V032JcYJikqXhkge2AFW-NDbL0v/s1600/vbawarnings.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWPSoy7g1EFJvLupKgznZDoJGXnCIakhBRekoqX2RMexRympf6ODdgTlSDsPE0PnHFEn2KrhZWrWnqSnrDhxG-aGmOYayDxbIP85PUzVe3UotTFQPc4V032JcYJikqXhkge2AFW-NDbL0v/s1600/vbawarnings.PNG" /></a></div>
<br />
Based on my testing with Word 2015, these are the Macro Settings and corresponding values for the registry flag:<br />
<br />
<ul>
<li>Disable all macros without notification : 4</li>
<li>Disable all macros with notification: 2</li>
<li>Disable all macros except digitally signed macros: 3</li>
<li>Enable all macros: 1</li>
</ul>
<br />
I believe these setting are also<a href="https://technet.microsoft.com/en-us/library/cc179039.aspx" target="_blank"> affect by a GPO</a>, but I have not been able to confirm this yet through testing.<br />
<br />
My testing was done using Office 2015 on Windows 7 and Office 2010 on Windows 10. These setting may also apply to Excel, Access and PowerPoint, but I have not tested these. <br />
<br />
<br />
So, to summarize:<br />
<br />
1) These artifacts may remain after the malicious document has been removed. They may also be shown in your timeline if you are using a tool like<a href="https://github.com/keydet89/Tools" target="_blank"> regtime.exe</a> to add registry keys into your timeline.<br />
<br />
2) If there is an entry for a document under Trusted Records, this does not necessarily mean that macros were enabled. The flag needs to be checked to make that determination.<br />
<br />
3) If a document does not appear under this key, this does not mean that the macros were not able to run. They could still have ran if the default setting was altered to enable all macros by default.<br />
<br />
<br />
<br />
<br />
Additional Resources:<br />
<br />
<a href="http://forensicartifacts.com/2012/07/ntuser-trust-records/" target="_blank">NTUSER Trust Records</a><br />
<br />
<a href="https://technet.microsoft.com/en-us/library/cc179039.aspx" target="_blank">Plan and configure Trusted Locations settings for Office 2013</a><br />
<br />
<a href="http://windowsir.blogspot.com/2013/07/howto-determine-user-access-to-files.html" target="_blank">HowTo: Determine User Access To Files</a><br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com1tag:blogger.com,1999:blog-2456910508235577340.post-50028234885336345252015-09-20T19:56:00.001-07:002016-05-30T16:59:22.281-07:00Who's your Master? : MFT Parsers Reviewed<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiKUbO2tB5-gUQBcOEMxjVeAFw0NrCIJmwJYAyuFcXvThVMoLwp2B77Jrl95qfhZwmWQbsqe0c2cQgtfEK8tq1Bq46neIbbNoQEszBFSetOtwisozh9JHbSzsP8EqA3D272uSohG99a_KG/s1600/kung-fu-mft.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiKUbO2tB5-gUQBcOEMxjVeAFw0NrCIJmwJYAyuFcXvThVMoLwp2B77Jrl95qfhZwmWQbsqe0c2cQgtfEK8tq1Bq46neIbbNoQEszBFSetOtwisozh9JHbSzsP8EqA3D272uSohG99a_KG/s400/kung-fu-mft.png" width="350" /></a></div>
The Master File Table (MFT) contains the information related to folders and files on an NTFS system. Brian Carrier (2005) stated “The Master File Table is the heart of NTFS because it contains the information about all files and directories” (p. 274) Many of the forensics tools such as EnCase, FTK and X-Ways parse the MFT to display the file and folder structure to the user.<br />
<br />
During Incident Response, there could be hundreds if not thousands of computers to examine. A way to quickly review these systems for Indicators of Compromise (IOCs) is to grab the MFT file rather than take a full disk image. The MFT file is much smaller in size than a disk image and can be parsed to show existing as well as deleted files on a system.<br />
<br />
During a case, I noted some anomalies with a tool that I use to accomplish this task, <a href="https://github.com/dkovar/analyzeMFT" target="_blank">AnalyzeMFT</a>. This led me to do some testing and verification of several MFT parsers – and I was a little surprised with the results. Foremost, I would like to say that I am appreciative to all the authors of these tools. My intent with this post is to draw attention to understanding the outputs of these tools so that the examiner can correctly interpret the results.<br />
<br />
Many of the differences and issues arose due the handling of deleted files. The documentation of one of the tools I tested, MFTDump, explains the issues with deleted files in the MFT:<br />
<blockquote class="tr_bq">
"Since MFTDump only has access to the $MFT file, it is not possible to “chase‟ down deleted files through the $INDEX_ALLOC structures to determine if the file is an orphan. Instead, the tool uses the resident $FILE_NAME attribute to determine its parent folder, and follows the folder path to the root folder. In the case of deleted files, this information may or may not be accurate. To determine the exact status of a deleted file, you need to analyze the file system in a forensic tool."</blockquote>
Some of the tools did not notify the examiner that the file path associated with the deleted file may be incorrect – which could lead to some false conclusions.<br />
<br />
There are a lot of tools that parse the MFT. For this testing, I focused on tools that are free, command line and output the results into <a href="http://wiki.sleuthkit.org/index.php?title=Body_file" target="_blank">Bodyfile</a> format. The reason I chose to do this is that when I parse the MFT, I am using it to create a timeline, usually in an automated fashion. The one exception to this was the tool <a href="http://malware-hunters.net/all-downloads/" target="_blank">MFTDump</a>. The output was a TSV file that I<a href="https://github.com/mdegrazia/mft-parse" target="_blank"> wrote a parser</a> for that converted it into Bodyfile format.<br />
<br />
There were four “things” that I was checking each tool for: File Size, Deleted Files, Deleted File Paths and Speed. This criteria may not be important to everyone, but I’ll explain why these are important to me.<br />
<ol>
<li><u><b>File Size</b></u><br /> When looking for IOC’s, file size can be used to distinguish a legitimate file from malware that has the same name. It could also be used in lieu of file hashes. Instead of hashing every file on the computer which can be time consuming, the hashed file's size can be used to do a comparison of the MFT file sizes to flag suspect files (thanks to @rdormi for that idea)</li>
<li><u><b>Deleted Files</b></u><br />MFT records can contain deleted file information. Does the output show deleted files? In some cases the attacker’s tools and malware have been removed from the system, so being able to see deleted files is nice.</li>
<li><u><b>Deleted File Paths</b></u><br />Is the tool able to resolve and display any portion of the previous file path for the deleted file? Knowing the parent path helps give context to the file. For example, it may be located under a user account, or a suspicious location, like a temp folder<b>.</b><u><b><br /></b></u></li>
<li><u><b>Speed</b></u><br />If I am processing thousands of machines, I need a tool that will parse the MFT relatively quickly. 10 minutes per machine or 1 hour per machine can make a big difference.<br /><u><br /></u></li>
</ol>
<h3>
Findings </h3>
The tools I tested were <a href="https://github.com/dkovar/analyzeMFT" target="_blank">AnalyzeMFT</a>,<a href="https://code.google.com/p/log2timeline/" target="_blank"> log2timeline.pl</a>, <a href="http://www.williballenthin.com/forensics/mft/list_mft/" target="_blank">list-mft</a> and <a href="http://malware-hunters.net/all-downloads/" target="_blank">MFTDump</a>. Below is a summary of the findings. Further below, I explain the results in more detail, along with some sample data.<br />
<br />
<table style="width: 100%;">
<tbody>
<tr>
<td><b>AnalyzeMFT</b><br />
<ol>
<li>Many files, both deleted and existing, show an incorrect file size of 0</li>
<li>Deleted files were not designated as deleted in the output</li>
<li>Deleted files where prepended with incorrect file paths </li>
<li>Time to parse MFT: 11 minutes</li>
</ol>
</td>
<td rowspan="4"><div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTZavVQPNlpkOALiaRwT0I3OlCnRlarYWwGzhO0dqg3mC9DgGVMiIkYsRgGsL8PY0H_EcjkKhPRUs-lPPc2KEq48X3dMaG2MOiTfqxHLe9bOxBN4paeySKwdemQDgyc3zcDZOX1UdemzjE/s1600/speed-racer-MFT.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="236" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTZavVQPNlpkOALiaRwT0I3OlCnRlarYWwGzhO0dqg3mC9DgGVMiIkYsRgGsL8PY0H_EcjkKhPRUs-lPPc2KEq48X3dMaG2MOiTfqxHLe9bOxBN4paeySKwdemQDgyc3zcDZOX1UdemzjE/s400/speed-racer-MFT.png" width="400" /></a></div>
</td>
</tr>
<tr>
<td><b> List-mft</b><br />
<ol>
<li><b> </b>File sizes were shown in the output</li>
<li> Deleted files were designated as deleted</li>
<li> No file paths were shown for deleted files</li>
<li>Time to parse MFT: 1 hour, 49 minutes<div class="separator" style="clear: both; text-align: center;">
</div>
</li>
</ol>
</td>
</tr>
<tr>
<td><b>Log2timeline.pl </b><br />
<ol>
<li>No file sizes were shown in the output</li>
<li>Deleted files were designated as deleted</li>
<li>Deleted files were shown with correct file paths</li>
<li>Time to parse MFT: 39 minutes</li>
</ol>
</td>
</tr>
<tr>
<td><b>MFTDump </b><br />
<ol>
<li> File sizes were shown in the output</li>
<li>Deleted files were designated as deleted</li>
<li>Deleted files were enclosed with ‘?’ to alert the examiner that file paths may be inaccurate</li>
<li>Time to parse MFT: 7 minutes</li>
</ol>
</td>
</tr>
</tbody></table>
Please note, I did not cross reference and verify <i>every single file</i> in the output. The observations made above were for the files that I reviewed.<br />
<br />
What does this mean, or why are these results important?<br />
<br />
<u><b>No file size reported</b></u><br />
The file size can help give context to a file. Having the file size can help determine if a file is suspect or not. If no file size is provided, this context is lost.<br />
<br />
<u><b>'0’ File size reported</b></u><br />
The incorrect file size of ‘0’ can be misleading to an investigator. Take into consideration a RAM
scraper output file. If an examiner is checking various systems and they see a file size of ‘0’, they might think the file is empty, when in fact, it could have thousands of credit card numbers written to it.<br />
<br />
<u><b>Files are not being reported/noted as deleted</b></u><br />
Since there is no designation that the file is deleted, malware might appear to exist on a system, when in fact, it has been deleted. A suspect may have deleted a file and it is still showing as active in the output.<br />
<u><br /></u>
<u><b>Deleted files are being associated with the wrong parent path</b></u><br />
As noted above, due to issues with looking up the parent folder for deleted files, incorrect file paths were found to be prepended to deleted files. Even though a portion of the path may be correct, the prepended path could cause the examiner to draw an incorrect conclusion.<br />
<br />
For example, many times a malware file will have a legitimate windows system name, such as svchost.exe. What flags the file as suspicious is where it was/is located. If the parent path is reportedly incorrectly, a malicious file may be missed. Or, a file may my attributed to an incorrect user account because the path is listed incorrectly.<br />
<br />
<h3>
<b>Conclusion </b></h3>
Based on my testing and criteria, MFTDump seems to be the best fit for my process. It contains the file sizes, and designates between
an active file and a deleted file. In the event that it recovers a file path
for a deleted file, it lets the examiner know that it might be inaccurate by making
a notation in the output. If any important files are found using any of these
tools, it would be prudent for the examiner to verify with a full disk image. <br />
<br />
<h3>
<b>Sample Test Data</b> </h3>
Below, I show some examples from the output for each tool. Although I did some testing and verification, it is up to each examiner to test their tools – I accept no liability or responsibility for using these tools and relying on my results. For demonstrative purposes only. :) <br />
<br />
I used FLS from the Sleuthkit and X-Ways to check a deleted file. I then compared how this deleted file was handled with the different tools. I also used <a href="https://github.com/keydet89/Tools" target="_blank">Harlan Carvey’s tools</a> (bodyfile.exe and parse.exe) to convert the bodyfile generated by the tool into TLN format for readability. <br />
<br />
The deleted file I reviewed was “048002.jpg”. The path was shown as C:/$OrphanFiles/Pornography/048002.jpg (deleted) in both FLS and X-Ways.<br />
<br />
Each of the outputs were grepped for the file 048002.jpg, and the entries located are displayed below in TLN format. I omitted the "Type" (File), "Host" (Computer1) and "User" (blank) columns in order to better display the results.<br />
<br />
I have also included how long each process took. The system I used was Windows 7 with an Intel i7 and 16GB of RAM. The size of the MFT was about 1.8GB (which is much larger then most systems I process) <br />
<b> </b><br />
<b>FLS Output</b><br />
fls -m C: -f ntfs -r \\.\[Mounted Drive] >> C:\path\to\bodyfile<br />
<br />
<table border="1" cellpadding="2" cellspacing="2" style="width: 100%px;">
<tbody>
<tr>
<td valign="top"><b>Date</b></td>
<td valign="top"><b>Description</b></td>
</tr>
<tr>
<td valign="top">2076-11-29 08:54:34</td>
<td valign="top">MA.B [4995]
C:/$OrphanFiles/Pornography/048002.jpg (deleted)</td>
</tr>
<tr>
<td valign="top">2014-01-11 01:25:45</td>
<td valign="top">..C. [4995]
C:/$OrphanFiles/Pornography/048002.jpg (deleted)</td>
</tr>
<tr>
<td valign="top">2013-10-28 20:38:37</td>
<td valign="top">MACB [124] C:/$OrphanFiles/Pornography/048002.jpg ($FILE_NAME) (deleted)</td>
</tr>
</tbody>
</table>
<br />
FLS was used as the baseline for the test, and the output was verified with X-Ways. It shows the file as a deleted Orphan file, with a partial recovered directly listing of "/Pornography/048002.jpg". According to The <a href="http://wiki.sleuthkit.org/index.php?title=Orphan_Files" target="_blank">Sleuthkit</a> documentation on orphan files:<br />
<blockquote class="tr_bq">
"Orphan files are deleted files that still have file metadata in the file
system, but that cannot be accessed from the root directory."</blockquote>
Fls took about 20 minutes to run accross the mounted image.<br />
<br />
<b>AnalyzeMFT Output</b><br />
analyzeMFT.py -f<b> </b>"C:\path\to\$MFT" -b "C:\path\to\output\bodyfile.txt" --bodyfull -p<br />
<br />
<table border="1" cellpadding="2" cellspacing="2" style="width: 100%px;">
<tbody>
<tr>
<td valign="top"><b>Date</b></td>
<td valign="top"><b>Description</b></td>
</tr>
<tr>
<td valign="top">2013-10-28 20:38:37</td>
<td valign="top">MACB [0]
/Users/SpeedRacer/AppData/Roaming/Scooter Software/Beyond
Compare 3/BCState.xml/Pornography/048002.jpg</td>
</tr>
</tbody>
</table>
<br />
AnalyzeMFT showed 0 for the file size. It had no designation in the output that flags if the file is deleted or active. Although it was able
to recover the deleted file path "/Pornography/", it prepended the file
path with a folder that currently exists on the system rather then identify it as an Orphan file.<br />
<br />
This makes it appear to the examiner that this is an active file, under the location "Users/SpeedRacer/AppData/Roaming/Scooter Software/Beyond
Compare 3/BCState.xml/Pornography", when in fact, it is a deleted Orphan file. <br />
<br />
During my review of the outputs, I noticed quite a few files were showing an incorrect file size of '0', including active files. In the review of the open issues on github, these issues appear to have been noted.<br />
<br />
I also ran AnalyzeMFT with the default output, a csv file. In this output, the file did have a flag designating it as deleted, however, the bodyfile format does not. <br />
<br />
<b>Log2Timeline.pl Output</b><br />
log2timeline -z local -f mft -o tln -w /path/to/bodyfile.txt<br />
<br />
<b></b><br />
<table border="1" cellpadding="2" cellspacing="2" style="width: 100%px;">
<tbody>
<tr>
<td valign="top"><b>Date</b></td>
<td valign="top"><b>Description</b></td>
</tr>
<tr>
<td valign="top">2014-01-11 01:25:45</td>
<td valign="top">FILE,-,-,[$SI ..C.] /Pornography/048002.jpg
(deleted)|UTC| inode:781789</td>
</tr>
</tbody>
</table>
<br />
The “old” version of log2timeline has an –f mft option that parses an
MFT file into bodyfile format. The “new” version of log2timeline with Plaso
does not have the option to parse the MFT separately (at least I coudnt find it.). log2timeline.pl was run from a SIFT Virtual Machine. For the VM, I gave the VM about 11GB of RAM, and 6 CPUs. With this setup, it took about 39 minutes to parse the MFT. <br />
<br />
No
file size was provided in the log2timeline for any files. The file is
flagged as deleted, and includes the correct partial recovered path /Pornography/". Out off all the MFT tools I tested, this one most accurately depicts the deleted file path. However, it's interesting to note that it did not include the FileName attribute.<br />
<b><br /></b>
<b>list-mft Output</b><br />
list-mft.py "C:\path\to\$MFT" >> "C:\path\to\output\bodyfile.txt"<br />
<br />
<table border="1" cellpadding="2" cellspacing="2" style="width: 100%px;">
<tbody>
<tr>
<td valign="top"><b>Date</b></td>
<td valign="top"><b>Description</b></td>
</tr>
<tr>
<td valign="top">2014-01-11 01:25:45</td>
<td valign="top">,..C. [4995] \\$ORPHAN\048002.jpg (inactive)</td>
</tr>
<tr>
<td valign="top">2013-10-28 20:38:37</td>
<td valign="top">,MACB [4995] \\$ORPHAN\048002.jpg (filename,
inactive)</td>
</tr>
</tbody>
</table>
<br />
list-mft provided the file size, and a designation that the file was deleted (inactive). It also identified the file as an Orphan, however, it did not recover the partial path of /Pornography/. This may be important as the partial path can help provide context for the deleted file.<br />
<br />
This program took the longest to run at 1 hour and 49 minutes. There is a -c, cache option that can be configured. This can be increased for better performance, however, I just used the default settings.<br />
<br />
<b>MFTDump Output</b><br />
mftdump.exe "C:\path\to\$MFT" <b>/o </b>"C:\path\to\output\mftdump-output.txt"<br />
<br />
<table border="1" cellpadding="2" cellspacing="2" style="width: 100%px;">
<tbody>
<tr>
<td valign="top"><b>Date</b></td>
<td valign="top"><b>Description</b></td>
</tr>
<tr>
<td valign="top">2076-11-29 08:54:34</td>
<td valign="top">MA.B [4995]
?\Users\SpeedRacer\AppData\Roaming\Scooter Software\Beyond
Compare 3\BCState.xml\Pornography\048002.jpg?(DELETED)</td>
</tr>
<tr>
<td valign="top">2014-01-11 01:25:45</td>
<td valign="top">..C. [4995]
?\Users\SpeedRacer\AppData\Roaming\Scooter Software\Beyond
Compare 3\BCState.xml\Pornography\048002.jpg?(DELETED)</td>
</tr>
<tr>
<td valign="top">2013-10-28 20:38:37</td>
<td valign="top">MACB [4995]
?\Users\SpeedRacer\AppData\Roaming\Scooter Software\Beyond
Compare 3\BCState.xml\Pornography\048002.jpg? (DELETED)(FILENAME)</td>
</tr>
</tbody>
</table>
<br />
The file sizes are displayed, and a designation is included showing that the file has been deleted. Deleted files were enclosed with ‘?’ to alert the examiner that file paths may be incorrect. This tool ran the fastest, clocking 7 minutes for a 1.8 GB MFT file. The output from this tool as a TSV file. I wrote a python script to parse it into bodyfile format.<br />
<b></b><br />
To keep this post relativity short, I just demonstrated the output for one file, however, I used the same process on several files and the results were consistent. Whatever tool an examiner chooses to use will depend on their particular needs. For example, an examiner may not be interested in file sizes, and in this case they may choose to use log2timeline. However, if speed is an issue, MFTDump might make more sense. As long as the examiner knows what information the output is portraying, and can verify the results independently, any of these tools can get the job done.<br />
<br />
Carrier, B. (2005).<i> File System Forensic Analysis</i>. Upper Saddle River, NJ: Pearson Education <br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com5tag:blogger.com,1999:blog-2456910508235577340.post-54359553520447952642015-06-22T07:34:00.000-07:002015-06-22T07:34:44.423-07:00SQLite Deleted Data Parser Update - Leave no "Leaf" unturnedOne of the things I <i><b>love</b></i> about open source is that people have the ability to update and share code. Adrian Long, aka @Cheeky4n6Monkey, did just that. Based upon some research, he located additional deleted data that can be harvested from re-purposed SQLite pages - specifically the Leaf <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhp_4Th3ppCZ_rZkg__QGxh9DUe3qOgTzP1g5G-4OcQ6JzgLJXrWf5aAUXVlZRpemb4EnsqDjVjxkeV_j3EG435-p1Qlu5OgcZHEwzP0ol2F0feZvYIBlVPlIM6RqiNpyyciS0DTKZC2fm4/s1600/sqlite-leaf.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="283" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhp_4Th3ppCZ_rZkg__QGxh9DUe3qOgTzP1g5G-4OcQ6JzgLJXrWf5aAUXVlZRpemb4EnsqDjVjxkeV_j3EG435-p1Qlu5OgcZHEwzP0ol2F0feZvYIBlVPlIM6RqiNpyyciS0DTKZC2fm4/s320/sqlite-leaf.png" width="320" /></a></div>
Table B-Tree page type. He updated my code on GitHub and BAM!! just like that, the SQLite Deleted Data parser now recovers this information.<br />
<br />
He has detailed all the specifics and technical goodies in a <a href="http://cheeky4n6monkey.blogspot.com/">blog post</a>, so I won't go into detail here. It involved a lot of work and I would like to extend a huge thank you to Adrian for taking the time to update the code and for sharing his research.<br />
<br />
You can download the most recent version on my <a href="https://github.com/mdegrazia/SQLite-Deleted-Records-Parser">GitHub page</a>. I've also update the command line and GUI to support the changes as well.Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com0tag:blogger.com,1999:blog-2456910508235577340.post-4609296569265484822015-06-09T07:35:00.000-07:002015-06-09T07:35:25.919-07:00Does it make sense?<br />
Through all my high school and college math classes, my teachers always taught me to step back after a problem was completed and ask if the answer made sense. What did this mean? It meant don't just punch numbers into the calculator, write the answer, and move on. It meant step back, review the problem, consider all the known information and ask, "Does the answer I came up with make sense?"<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUZ78x9MoDEK8Y7EQ9_nOgvK0Osd98lxk12ggIkVyce77H7n2b5Pa2PDExv1UbC3ItJ4qnLHxTHuJfCupnzDO1A-jXnkoAsMJ6knbgs-_aHGO4_Ys4ZBTnZgTHhugzil6jZnkLdp9XgcRm/s1600/right-triangle.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUZ78x9MoDEK8Y7EQ9_nOgvK0Osd98lxk12ggIkVyce77H7n2b5Pa2PDExv1UbC3ItJ4qnLHxTHuJfCupnzDO1A-jXnkoAsMJ6knbgs-_aHGO4_Ys4ZBTnZgTHhugzil6jZnkLdp9XgcRm/s1600/right-triangle.png" /></a></div>
Take for instance the Pythagorean Theorem. Just by looking at the picture, I can see that C should be longer than A or B. If my answer for C was smaller than A or B, I would know to recheck my work.<br />
<br />
Although the above example is relatively simple, this little trick applied to more complicated math problems, and many times it helped me catch incorrect answers.<br />
<br />
But what does this have to do with DFIR? I believe the same principle can be applied to investigations. When looking at data, sometimes stepping back and asking myself the question, "Does what I am looking at make sense?" has helped me locate issues I may not have otherwise caught.<br />
<br />
I have had a least a couple of DFIR situations where using this method paid off.<br />
<br />
<b>You've got Mail...</b><br />
I was working a case where a user had Mozilla Thunderbird for an email client. I parsed the email with some typical forensic tools and begin reviewing emails.<br />
<br />
While reviewing the output, I noticed it seemed pretty sparse, even though the size of the profile folder was several gigs. This is where I stepped back and asked myself, <i>does this make sense</i>? It was a large profile, yet contained very few emails. This led to my research and eventual <a href="http://az4n6.blogspot.com/2014/04/whats-word-thunderbird-parser-that-is.html">blog post</a> on the Thunderbird MBOXRD file format. Many of the programs were just parsing out the MBOX format and not the MBOXRD format, and thus, missing <i>lots</i> of emails. Had I just accepted the final output of the program, I would have missed a lot of data. <br />
<br />
<b>All your files belong to us..</b>.<br />
Many times I will triage a system either while waiting for an image to complete, or as an alternate to taking an image. This is especially useful when dealing with remote systems that need to be looked at quickly. Exporting out the MFT and other files such as the Event Logs and Registry files results in a much smaller data set than a complete image. These artifacts can then be used to create a mini-timeline so analysis can begin immediately (see Halan's post <a href="http://windowsir.blogspot.com/2015/04/micro-mini-timelines.html">here</a> for more details on creating mini-timelines). <br />
<br />
To parse the MFT file into timeline format, I use a tool called <a href="https://github.com/dkovar/analyzeMFT">Analyze MFT</a> to provide a bodyfile. Once the MFT is in bodyfile format, I use Harlan Carvey's parse.pl to convert it into TLN format and add it into the timeline.<br />
<br />
While working a case, I created timelines using the above method for several dozen computers. After the timelines were created, I grepped out various malware references from these timelines. While reviewing the results, I noticed many of the malware files had a file size of zero. Weird. I took a closer look and<b><i> </i></b>noticed <i>ALL</i><b><i> </i></b>the malware files contained a file size of zero. Hmmm.. what did that mean? What are the chances that <i>ALL</i> of those files would have a zero file size??? Since the full disk images were not available, validateing this information with the actual files was not an option. But I stepped backed and asked myself, given what I knew about my case and how the malware behaved.. <i>does that make sense?</i><br />
<br />
So I decided to "check my work" and do some testing with Analyze MFT. I created a virtual machine with Windows XP and exported out the MFT. I parsed the MFT with Analyze MFT and began looking at the results for files with a zero file size.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2XerUWgQDY5iITmI6-SHPRrgZdDjIhMRJhCTh-Md-Lh0fjyUef5T2RvKMh49E60jPf74jxPwhbphGBHMipkd3cbzMoutTKwxk55jiTZgPBdA15rzGBcmDYrznB4yiqlO175Nct3Vu7Rse/s1600/what+it+means.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="196" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj2XerUWgQDY5iITmI6-SHPRrgZdDjIhMRJhCTh-Md-Lh0fjyUef5T2RvKMh49E60jPf74jxPwhbphGBHMipkd3cbzMoutTKwxk55jiTZgPBdA15rzGBcmDYrznB4yiqlO175Nct3Vu7Rse/s320/what+it+means.jpg" style="cursor: move;" width="320" /></a>I noticed right away that all the prefetch files had a file size of zero, which is odd. I was able to verify that the prefetch files sizes were in fact NOT zero by using other tools to parse the MFT, as well as looking at the prefetch files themselves in the VM. My testing confirmed that Analyze MFT was incorrectly reporting a file size of zero for some files.<br />
<br />
After the testing I reached out to David Kovar, the author of Analyze MFT, to discuss the issue. I also submitted a bug to the github page.<br />
<br />
If I had not "checked my work" and assumed that the the file size of zero meant the files were empty, it could have led to an incorrect "answer".<br />
<br />
So thanks to those teachers that ground the "does it make sense" check into my head, as it has proved to be a valuable tip that has helped me numerous times (more so then the Pythagorean Theorem...)Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com1tag:blogger.com,1999:blog-2456910508235577340.post-90316892765871801562015-04-07T08:13:00.001-07:002015-04-07T08:13:12.150-07:00Dealing with compressed vmdk files<br />
Wherever I get vmdk files, I take a deep breath and wonder what issues might pop up with them. I recently received some vmkd files and when I viewed one of these in FTK Imager (and some other
mainstream forensic tools), it showed up as the dreaded "unrecognized file system".<br />
<br />
To verify that I had not received some corrupted files, I used the <a href="https://my.vmware.com/group/vmware/details?productId=285&downloadGroup=VSP510-VDDK-510">VMWares disk </a>utility to check the partitions in the vmdk file. This tool showed two volumes, so it appeared the vmdk file was not corrupted:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfkcSUdHO27TYyz7znhVEnuBFlnRu2GXMGuV7O232VO-Z6YFIyNyFxcmWmSTRMgd1KAPIe_velmnrBmZmfrtyoJLSa3RmhFRHhL30uGQrRhZvw2tmrozE4-m692thKZtzXiuF7ZhUlDUSd/s1600/vmware-mount.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgfkcSUdHO27TYyz7znhVEnuBFlnRu2GXMGuV7O232VO-Z6YFIyNyFxcmWmSTRMgd1KAPIe_velmnrBmZmfrtyoJLSa3RmhFRHhL30uGQrRhZvw2tmrozE4-m692thKZtzXiuF7ZhUlDUSd/s1600/vmware-mount.PNG" height="204" width="640" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
When I tired to mount the vmdk file using vmware-mount, the drive mounted, but was not accessible. A review of their documentation, specifically the <a href="https://pubs.vmware.com/vsphere-50/index.jsp#com.vmware.vddk.utils.doc_50/diskutils_mount.4.2.html">limitation</a> section, pointed out that the utility could not mount compressed vmdk files:<br />
<br />
<i>You cannot mount a virtual disk if any of its files are encrypted, <span style="background-color: yellow;">compressed</span>, or have read-only permissions. Change these attributes before mounting the virtual disk</i><br />
<br />
Bummer. It appeared I had some compressed vmdk files. <br />
<br />
So after some Googling and research, I found a couple different ways to deal with these compressed vmdk files - at least until they are supported by the mainstream forensic tools. The first way involves converting the vmdk file, and the second way is by mounting it in Linux.<br />
<br />
Which method I choose ultimately depend on my end goals. If I want to bring the file into one of the mainstream forensics tools, converting it into another format may work the best. If I want to save disk space, time and do some batch processing, mounting it in Linux may be ideal.<br />
<br />
One of the first things I do when I get an image is create a mini-timeline using fls and some of Harlan's tools. Mounting the image in Linux enables me to run these tools without the additional overhead of converting the file first.<br />
<br />
<b>Method 1: "Convert"</b><br />
<br />
The first method is to "convert" the vmdk file. <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjvjTrnTTwKkIavoXwGjKeQDOPmmpV9b1U9EeFdoWe8vtdXdZSihDTR-DJaeEKKL_F9mlGJsVkm0BvZXps5Qr6WVLdX4hdIZLM5NUNoo5PJ6TOLGfOUqTrNkbW5lDwQeBCTPDAL_dktZ8N/s1600/convert.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjvjTrnTTwKkIavoXwGjKeQDOPmmpV9b1U9EeFdoWe8vtdXdZSihDTR-DJaeEKKL_F9mlGJsVkm0BvZXps5Qr6WVLdX4hdIZLM5NUNoo5PJ6TOLGfOUqTrNkbW5lDwQeBCTPDAL_dktZ8N/s1600/convert.jpg" height="258" width="320" /></a></div>
I'm using "quotes" because my preferred method is to "convert" it right back to the vmdk file format, which in essence, decompresses it.<br />
<br />
The vmdk file format is usually much smaller then the raw/dd image and appears to take less time time to "convert".<br />
<br />
I used the the free<a href="https://www.virtualbox.org/manual/UserManual.html"> VBoxManger.exe</a> that comes with VirtualBox. This is a command line tool located under C:\Program Files\Oracle\VirtualBox. This tool give you the option to convert the compressed vmdk (or a regular vmkd) into several formats: VHD, VDI, VMDK and Raw. The syntax is:<br />
<br />
VboxManage.exe clonehd "C:\path\to\compressed.vmkd" "C:\path\to\decompressed.vmdk" --format VMDK.
<br />
<br />
It give you a nice status indicator during the process:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ1imFi_QypZgiLIVZEv1xvUZrzmAyN53lbvIFkw4p6ptO1e5Yu7bhqBhbWMaWUqLMMGK5c_TouR_8xVQ3Y4iwHaw5v_B-ar-dWRFyHslum-Z7RXWUCiSPJvZHraQJ0cugkbbCvxKgi8Xh/s1600/vboxmanager.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ1imFi_QypZgiLIVZEv1xvUZrzmAyN53lbvIFkw4p6ptO1e5Yu7bhqBhbWMaWUqLMMGK5c_TouR_8xVQ3Y4iwHaw5v_B-ar-dWRFyHslum-Z7RXWUCiSPJvZHraQJ0cugkbbCvxKgi8Xh/s1600/vboxmanager.PNG" height="162" width="640" /></a></div>
<br />
<br />
Now the file is in a format that can worked with like any other normal vmdk file.<br />
<br />
<b>Method 2: Mount in Linux</b><br />
<br />
This is the method that I prefer when dealing with LOTS of vmdk files. This method uses Virtual Box Fuse, and does not require you to decompress/convert the vmkd first.<br />
<br />
I had a case involving over 100 of these compressed files. Imagine the overhead time involved with converting 100+ vmdk files before you can even begin to work with them. This way, I was able to write a script to mount each one in Linux, run fls to create a bodyfile, throw some of Harlan's parsers into the mix, and was able to create a 100+ mini-timelines pretty quickly.<br />
<br />
There is some initial setup involved but once that's done, it's relatively quick and easy to access the compressed vmdk file.<br />
<br />
I'll run though how to install Virtual Box Fuse, how to get the compressed vmkd file mounted, then run fls on it.<br />
<br />
<b>1)Install VirtualBox:</b><br />
<br />
<div class="code">
sudo apt-get install virtualbox-qt</div>
<br />
<b>2) Install Virtual Box Fuse</b>. It is no longer in the app repository, so you will need to download and install the .deb file - don't worry, it's pretty easy, no compiling required :)<br />
<br />
Download the .deb from from <a href="https://launchpad.net/ubuntu/raring/+package/virtualbox-fuse">Launchpad</a> under "Published Versions". Once you have it downloaded, install it by typing:<br />
<br />
<div class="code">
sudo dpkg -i --force-depends virtualbox-fuse_4.1.18-dfsg-1ubuntu1_amd64.deb</div>
<br />
Note - this version is not compatible with Virtual Box v. 4.2. At the time of this writing, when I installed Virtual Box on my Ubuntu distro, it was version 4.1 and worked just fine. If you have a newer version of virtual box, it will still work - you just unpack the .deb file and run the binary without installing it. See the bottom of the thread <a href="http://askubuntu.com/questions/326060/how-to-mount-a-vdi-file-in-ubuntu-13-10">here</a> for more details.<br />
<br />
<b>3)Mount the compressed VMDK file read-only</b><br />
<br />
<br />
<div class="code">
vdfuse -r -t VMDK -f /mnt/evidence/compressed.vmdk /mnt/vmdk</div>
<br />
This will created a device called "EntireDisk" and Parition1, Parition2 etc. under /mnt/vmdk<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFpJlGNM2CNf5Da_EdBi5WHJPibc0-qN06daGzp2Agj0GjkCr4hdvMd8e1WY_ELigw7W8zdFmF4BAFlD-5IuFScGon50R37vtEOEscqr3qmKIQ5VXHMH-p_hnSKLcdZ7DxgQhTqSP9VBup/s1600/virutalbox-fuse.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFpJlGNM2CNf5Da_EdBi5WHJPibc0-qN06daGzp2Agj0GjkCr4hdvMd8e1WY_ELigw7W8zdFmF4BAFlD-5IuFScGon50R37vtEOEscqr3qmKIQ5VXHMH-p_hnSKLcdZ7DxgQhTqSP9VBup/s1600/virutalbox-fuse.PNG" height="186" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
(even though I got this fuse error - everything seems to work just fine)<br />
<br />
At this point and time, you can use fls to generate a bodyfile. fls is included in the Sleuth Kit, and is installed on SIFT by default. You may need to specify the offset for your partition. Run mmls to grab this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjr7g8ZsUNeg6bbaispz9eOSVR82QtXBXhvHWrnEPClZgUvnop_IiBi8jvfzD8Ff_AHWxm5AjeEG6vTa58Wli_nkRMdpOBMS89HArN3Thlvz-JbKPiRYcsaxFwAOoVTzGJiZ7SfGgy2Zuzx/s1600/mmls.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjr7g8ZsUNeg6bbaispz9eOSVR82QtXBXhvHWrnEPClZgUvnop_IiBi8jvfzD8Ff_AHWxm5AjeEG6vTa58Wli_nkRMdpOBMS89HArN3Thlvz-JbKPiRYcsaxFwAOoVTzGJiZ7SfGgy2Zuzx/s1600/mmls.PNG" height="246" width="640" /></a></div>
<br />
Now that we have the offsets, we can run fls to generate a bodyfile:<br />
<br />
<div class="code">
fls -r -o 2048 /mnt/vmdk/EntireDisk -m C: >> /home/sansforensics/win7-bodyfile <br />
fls -r -o 206848 /mnt/vmdk/EntireDisk -m C: >> /home/sansforensics/win7-bodyfile</div>
<br />
<br />
Next, if you want access to the files/folders etc, you will need to mount the EntireDisk Image as an ntfs mount for each partition. This is assuming you have an Windows system - if not, adjust the type accordingly:<br />
<br />
Mount Partition 1, Offset 2048:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZ5ZSsJYUPzQywZeTKZAtoU7q0MOiP1wwHB24oLWjQiFSmJKrCd_APNnqKRODFPjTxS1KRtyrlidlLJfx-px1TN8hXjV_bCH3T2O13bCFO80oEGk7dC2W8ZPgeDUP-H9ruVAYUaYIJFSZr/s1600/partition1.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZ5ZSsJYUPzQywZeTKZAtoU7q0MOiP1wwHB24oLWjQiFSmJKrCd_APNnqKRODFPjTxS1KRtyrlidlLJfx-px1TN8hXjV_bCH3T2O13bCFO80oEGk7dC2W8ZPgeDUP-H9ruVAYUaYIJFSZr/s1600/partition1.PNG" height="138" width="640" /></a></div>
<br />
Mount Parition2, Offset 206848:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF3qy8936f1sJCZWzBAkFpvTpMUA72EURLyOR3n85Wh2rKxysMUiMoJsIoHqJLbKqgvod62IpLNfhEv3jw4jhZh7PbHu_jf-GtpJcJ8nO45DCasI_LLArl8V1MB1cktcMwMhUrYW8O0gnn/s1600/partition2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF3qy8936f1sJCZWzBAkFpvTpMUA72EURLyOR3n85Wh2rKxysMUiMoJsIoHqJLbKqgvod62IpLNfhEv3jw4jhZh7PbHu_jf-GtpJcJ8nO45DCasI_LLArl8V1MB1cktcMwMhUrYW8O0gnn/s1600/partition2.PNG" height="156" width="640" /></a></div>
<br />
<br />
There are multiple ways to deal with this compressed format, such as using VMWare or VirtualBox GUI to import/export the compresses file... these are just some examples of a couple of ways to do it. I tend to prefer command line options so I can script out batch files if necessary.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com4tag:blogger.com,1999:blog-2456910508235577340.post-76520575136396020722015-03-04T07:53:00.001-07:002015-03-04T21:30:08.422-07:00USN Journal: Where have you been all my life<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1UmTMddVopzCW0EiiR1aCTqRf26w4L4Dvqs06XIKofuM0ZMYnS_ELrJJXcU00bt6M8_PWHLhI_Xbsh2GodV50-DFizT8nOt7xRpr4fhvyNDEI5c1jiV9cb7ToJm4C8vyN4csOoRwmm-Pl/s1600/usnjournal.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1UmTMddVopzCW0EiiR1aCTqRf26w4L4Dvqs06XIKofuM0ZMYnS_ELrJJXcU00bt6M8_PWHLhI_Xbsh2GodV50-DFizT8nOt7xRpr4fhvyNDEI5c1jiV9cb7ToJm4C8vyN4csOoRwmm-Pl/s1600/usnjournal.png" height="286" width="320" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-UarMlOoVkSH4ldf6qmZwmgeXG_ek1vV1TpKSb8vsd_uxG3dqRi6Y1F2D-MhJU8gf_0diadnH7tpg8nzpR8McToiPJdI2xQ8BUpmuQY6Y-mNRF7FGoBoZISzVbWCfCjJ4gNz5vnQMrEKf/s1600/usnjournal.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><br /></a>
One of the goals of IR engagements is to locate the initial infection vector and/or patient zero. In order to determine this, timeline analysis becomes critical, as does determining when the malware was created and/or executed on a system.<br />
<br />
This file create time may become extremely critical if you're dealing with multiple or even hundreds of systems and trying to determine when and where the malware first made its way into the environment.<br />
<br />
But what happens when the malware has already been remediated by a Systems Administrator, deleted by an attacker, or new AV signatures are being pushed out, resulting in the malware being removed?<br />
<br />
Many of the residual artifacts demonstrate execution, however, it seems very few actually document <b><i>when</i></b> the file was created on the system. This is where the USN Journal recently helped me on a case. The USN Journal is by no means new.. but I just wanted to talk about a case study and share my experience with it, as I feel it's an often overlooked artifact.<br />
<br />
For purposes of demonstrative data, I downloaded and infected a Windows 7 VM with malware. This malware was from a phishing email that contained a zip file, voice#5734223.zip. This zip file contained a payload, voice.exe. For more details on this malware sample, check out<a href="http://malware-traffic-analysis.net/2015/01/27/index.html"> http://malware-traffic-analysis.net/2015/01/27/index.html</a><br />
<br />
So lets run through some typical artifacts that demonstrate execution along with the available timestamps and see what they do and don't tell us...<br />
<br />
<b>MFT</b><br />
The MFT contains the filesystem information - Modified, Accessed and Created dates, file size etc. However, a deleted file's MFT record may be
overwritten. If you're lucky, your deleted malware file will still have an entry
in the MFT - however, in my case this was not to be.<br />
<br />
<b>The ShimCache</b><br />
I won't go into to much detail here as Mandiant has a great<a href="https://www.mandiant.com/blog/leveraging-application-compatibility-cache-forensic-investigations/"> white paper</a> on this artifact. Basically, on most systems this artifact contains information on files that have been executed including path, file size and last modified date. I parsed this registry key with RegRipper, and located an entry for the test malware, voice.exe:<br />
<br />
C:\Users\user1\Downloads\voice#5734223\<span style="background-color: yellow;">voice.exe</span><br />
ModTime: Wed Jan 28 15:28:46 2015 Z<br />
Executed<br />
<br />
So what does this tell me? That voice.exe was in the Downloads path, was executed, and has a last modified date of 01/28/2015 - <sigh> no create date </sigh>. <br />
<br />
<b>UserAssist</b><br />
The User Assist is another awesome key... it displays the last time a file
was executed, along with a run count. Once again, using RegRipper to parse this I located an entry for the test malware:<br />
<br />
{CEBFF5CD-ACE2-4F4F-9178-9926F41749EA}<br />
Mon Feb 23 02:33:34 2015 Z C:\Users\user1\Downloads\voice#5734223\<span style="background-color: yellow;">voice.exe</span> (2)*<br />
<br />
By looking at this artifact, I can see that the file was executed twice - once on February 23rd, however, I don't know when the first time was. It could have been minutes, hours or days earlier. It still does very little to let me know when the file was created on the system, although I do know it should be sometime before this time stamp.<br />
<br />
<b>Prefetch File</b><br />
This is a great artifact that can show execution and even multiple times of execution. But what if the system is a Server, where prefetching may not be enabled? In my case, prefetching was enabled, but there was no prefetch file for the malware in question - at least that is what I thought until I checked the USN Journal. And, once again, it does not contain information related to when the file was created on the system.<br />
<br />
Ok, so I've reviewed a couple of typical artifacts that demonstrated that the<b> </b>malware executed (for more artifacts related to execution, check out this blog post by Mandiant <a href="https://www.mandiant.com/blog/execute/">"Did it Execute"</a>) With timeline analysis, I may even get an idea of when this file was most likely introduced on the system - however, a definitive create date would be nice to have. This brings me too.....the USN Journal.<br />
<br />
<b>USN Journal</b><br />
There are a couple of tools I use to parse the USN Journal. A quick, easy to use script is usnj.pl available from Harlan Carvey's <a href="http://code.google.com/p/winforensicaanalysis/downloads/detail?name=usnj.pl">GitHub page</a>.<br />
<br />
Parsing the USN Journal and looking for the malware in question, I see some interesting entries for voice.exe, highlighted in red below:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWkEQncovSJzGTgK0LaPe1hrxeYD5tqQsE-oQLJH_DM2Y2J3JeVLmPs2g2bO265ALNtVgazZYim-uZCcSe31t8MkxoqzLhHFRdz6qQ45B7GJYYOW61xk_mTN2BJ-DEiLwzlldt1eZUPHh9/s1600/usnjournal.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWkEQncovSJzGTgK0LaPe1hrxeYD5tqQsE-oQLJH_DM2Y2J3JeVLmPs2g2bO265ALNtVgazZYim-uZCcSe31t8MkxoqzLhHFRdz6qQ45B7GJYYOW61xk_mTN2BJ-DEiLwzlldt1eZUPHh9/s1600/usnjournal.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Sweeet! I now have a File_Create timestamp for voice.exe I can use in my timeline. I can also see that voice.exe was deleted relatively quickly ~ 30 seconds after it was created. This deletion occured about the same time when the prefetch file for it was created. This might be an indication that the malware deleted itself upon execution.<br />
<br />
It's also interesting to note that around the same time the prefetch file was created for voice.exe, a file called testmem.exe was created and executed (highlighted in yellow)..hmmmm.<br />
<br />
Time to dig deeper. For a little more detail on the USN Journal, there is the<a href="https://www.gettriforce.com/product/anjp-free/"> TriForce tool</a>. This tool processes three files: $MFT, $J and $Logfile. It then cross references these three files to build out some additional relationships. In my experience, this tool takes a bit longer to run. As you can see by the output below, I now have full file paths that may help add a little more context:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixgni3lAZrAgkwoFH9Em3oYE6RCaZzA-004MGHEAoGQdNpXd0tAXybbwuBpmXfzbTgVUxwhmmICXSxBuJa486-LS5kr6B0AUUaok1wmmwVtviUz1Udd0ahKWuzR3dmiHxhPUs05LNgQzMV/s1600/triforce.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixgni3lAZrAgkwoFH9Em3oYE6RCaZzA-004MGHEAoGQdNpXd0tAXybbwuBpmXfzbTgVUxwhmmICXSxBuJa486-LS5kr6B0AUUaok1wmmwVtviUz1Udd0ahKWuzR3dmiHxhPUs05LNgQzMV/s1600/triforce.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
That testmem.exe just became all that more suspicious due to it's location - a temp folder.<br />
<br />
By reviewing the USN Journal file, I was able to establish a create date of the malware. This create date located in the USN Journal gave me an additional pivot point to work with. This pivot point lead to some additional findings - a suspicious file, testmem.exe. (For some more on timeline<b> </b>pivoting, check out Harlan's post <a href="http://windowsir.blogspot.com/2015/03/how-do-you-do-analysis.html">here</a>). Not only did the create date help locate additional artifacts, but it can also help me home in on which systems may be patient zero. Malware may arrive on a system long before its executed.<br />
<br />
Just because it's not there - doesn't mean it didn't exist. For the case I was working, I did not have any existing prefetch files for the malware.
However, when I parsed the USN Journal, I saw the prefetch file for the
malware get created and deleted within the span of 40 minutes. I also saw some additional temporary files get created and deleted, some of which were not in the MFT. <br />
<br />
Alass, as sad as it is, my relationship with the USN Journal does have some shortcomings (and it's not <i>my</i> fault). <strike>Since it is a log file, it does "roll over"</strike> The USJ Journal can be limited in the amount of data that it holds - sometimes it seems all I see in it are Windows Update files. If a system is heavily used, and if the file was deleted months ago, it may no longer be in the USN Journal. However, all is not lost though,<a href="https://www.blogger.com/"><span id="goog_1526016754"></span> Rumor<span id="goog_1526016755"></span></a> has it that there may be some USN Journal files located in the Volume Shadow Copies so there may still be hope. Also, David Cowen points out the log file is not circular (as I once thought), and just frees the old pages to disk:<br />
<br />
"After talking to Troy Larson though I now understand that this behavior is due to the fact that the journal is not circular but rather pages are allocated and deallocated as the journal grows"<br />
<br />
This means that you can carve USN Journals records! Check out his <a href="http://hackingexposedcomputerforensicsblog.blogspot.com/2014/06/daily-blog-359-carving-usn-records.html">blog post</a> here for more information.<br />
<br />
Happy Hunting! <br />
<br />
<b>Additional reading on the USN Journal</b><br />
<br />
<a href="http://journeyintoir.blogspot.com/2013/01/re-introducing-usnjrnl.html">http://journeyintoir.blogspot.com/2013/01/re-introducing-usnjrnl.html</a><br />
<br />
<a href="http://forensicsfromthesausagefactory.blogspot.com/2010/08/usn-change-journal.html">http://forensicsfromthesausagefactory.blogspot.com/2010/08/usn-change-journal.html</a><br />
<br />
<a href="https://msdn.microsoft.com/en-us/library/windows/desktop/aa363798%28v=vs.85%29.aspx">https://msdn.microsoft.com/en-us/library/windows/desktop/aa363798%28v=vs.85%29.aspx</a><br />
<br />
*The run count number was modified from 1 to 2 on this output to illustrate a point.<br />
<br />
<b> </b><br />
<b> </b><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com5tag:blogger.com,1999:blog-2456910508235577340.post-43730049026307617372014-10-07T08:06:00.001-07:002014-10-07T08:06:56.572-07:00Timestomp MFT Shenanigans<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTiSWSnTCLiHfyP3pc0M0lWU8rfymIlaexxkRdD7khLIEzrleNHLDuSVsz4c1e1jL3Flt3Uex30fXpSmh2GQtcFEyH_T_pQqKh1bM3x6kDoie8zpkz7NXI7s6bRxgbJVSt7Psdrjz0jL4X/s1600/timestomp.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTiSWSnTCLiHfyP3pc0M0lWU8rfymIlaexxkRdD7khLIEzrleNHLDuSVsz4c1e1jL3Flt3Uex30fXpSmh2GQtcFEyH_T_pQqKh1bM3x6kDoie8zpkz7NXI7s6bRxgbJVSt7Psdrjz0jL4X/s1600/timestomp.png" height="320" width="173" /></a></div>
I was working a case a while back and I came across some malware that had time stomping capabilities. There have been numerous posts written on how to use the MFT as a means to determine if time stomping has occurred, so I won't go into too much detail here.<br />
<br />
<h3>
Time Stomping</h3>
<br />
Time Stomping is an Anti-Forensics technique. Many times, knowing when malware arrived on a system is a question that needs to be answered. If the timestamps of the malware has been changed, ie, time stomped, this can make it difficult to identify a suspicious file as well as answer the question, "When".<br />
<br />
Basically there are two "sets" of timestamps that are tracked in the MFT. These two "sets" are the $STANDARD_INFORMATION and $FILE_NAME. Both of these track 4 timestamps each - Modified, Access, Created and Born.
Or if you prefer - Created, Last Written, Accessed and Entry Modified
(To-mato, Ta-mato). The $STANDARD_INFORMATION timestamps are the ones normally viewed in Windows Explorer as well as most forensic tools.<br />
<br />
Most time stomping tools only change the $STANDARD_INFORMATION set. This means that by using tools that display both the $STANDARD_INFORMATION and $FILE_NAME attributes, you can compare the two sets to determine if a file may have been time stomped. If the $STANDARD_INFORMATION predates the $FILE_NAME, it is a possible red flag (example to follow).<br />
<br />
In my particular case, by reviewing the suspicious file's $STANDARD_INFORMATION and $FILE_NAME attributes, it was relatively easy to see that there was a mismatch, and thus, combined with other indicators, that time stomping had occurred. Below is an example of what a typical malware time stomped file looked like. As you can see, the <b>$STANDARD_INFORMATION</b> highlighted in red<b> </b>predates the <b>$FILE_NAME </b>dates (test data was used for demonstrative purposes)<b><br /></b><br />
<br />
System A \test\malware.exe<br />
<b><br /></b>
<b>$STANDARD_INFORMATION</b><br />
<span style="color: red;">M: Fri Jan 1 07:08:15 2010 Z</span><br />
A: Tue Oct 7 06:19:23 2014 Z<br />
C: Tue Oct 7 06:19:23 2014 Z<br />
<span style="color: red;">B: Fri Jan 1 07:08:15 2010 Z </span><br />
<br />
<b>$FILE_NAME</b><br />
M: Thu Oct 2 05:41:56 2014 Z<br />
A: Thu Oct 2 05:41:56 2014 Z<br />
C: Thu Oct 2 05:41:56 2014 Z<br />
B: Thu Oct 2 05:41:56 2014 Z<br />
<br />
However, on a couple of systems there were a few outliers where the time stomped malware $STANDARD_INFORMATION and $FILE_NAME modified and born dates matched:<br />
<br />
System B \test\malware.exe<br />
<br />
<b>$STANDARD_INFORMATION</b><br />
<span style="color: red;">M: Fri Jan 1 07:08:15 2010 Z</span><br />
A: Tue Oct 7 06:19:23 2014 Z<br />
C: Tue Oct 7 06:19:23 2014 Z<br />
<span style="color: red;">B: Fri Jan 1 07:08:15 2010 Z</span><br />
<br />
<b>$FILE_NAME</b><br />
<span style="color: red;">M: Fri Jan 1 07:08:15 2010 Z</span><br />
A: Thu Oct 2 05:41:56 2014 Z<br />
C: Thu Oct 2 05:41:56 2014 Z<br />
<span style="color: red;">B: Fri Jan 1 07:08:15 2010 Z</span><br />
<br />
Due to other indicators, it was pretty clear that these files were time stomped, however, I was curious to know what may have caused these dates to match, while all the others did not. In effect, it appeared that that Modified and Born dates were time stomped in both the $SI and $FN timestamps, however this was not the MO in all the other instances.<br />
<br />
Luckily, I remembered a <a href="http://windowsir.blogspot.com/2014/07/file-system-ops-effects-on-mft-records.html">blog post</a> written by Harlan Carvey where he ran various file system operations and reported the MFT and USN change journal output for these tests. I remembered that during one of his tests, some dates had been copied from the $STANDARD_INFORMATION into the $FILE_NAME attributes<i>. </i>A quick review of his blog post revealed the following had occurred during a rename operation . Below is a quote from Harlan's post:<br />
<br />
<i>"I honestly have no idea why the last accessed (A) and creation (B) dates
from the $STANDARD_INFORMATION attribute would be copied into the
corresponding time stamps of the $FILE_NAME attribute for a rename
operation"</i><br />
<br />
In my particular case it was not the accessed date and creation dates (B) that appeared to have been copied, but the <b>modified</b> and creation dates (B). Shoot.. not the same results as Harlan's test... but his system was Windows 7, and the system I was examining was Windows XP. Because my system was different, I decided to follow the procedure Harlan used and do some testing on a Windows XP to see what happened when I did a file rename.<br />
<br />
<h3>
Testing</h3>
<br />
My test system was Widows XP Pro SP3 in a Virtual Box VM. I used FTK Imager to load up the vmdk file after each test and export out the MFT record. I then parsed the MFT record with Harlan Carvey's mft.exe.<br />
<br />
First, I created "New Text Document.txt" under My Documents. As expected, all the timestamps in both the $STANDARD_INFORMATION and<i> </i>$FILE_NAME<i> </i>were the same<i>:</i><br />
<i><br /></i>
12591 FILE Seq: 1 Link: 2 0x38 4 Flags: 1 <br />
[FILE]<br />
.\Documents and Settings\Mari\My Documents\New Text Document.txt<br />
M: Thu Oct 2 23:22:05 2014 Z<br />
A: Thu Oct 2 23:22:05 2014 Z<br />
C: Thu Oct 2 23:22:05 2014 Z<br />
B: Thu Oct 2 23:22:05 2014 Z<br />
FN: NEWTEX~1.TXT Parent Ref: 10469 Parent Seq: 1<br />
M: Thu Oct 2 23:22:05 2014 Z<br />
A: Thu Oct 2 23:22:05 2014 Z<br />
C: Thu Oct 2 23:22:05 2014 Z<br />
B: Thu Oct 2 23:22:05 2014 Z<br />
FN: New Text Document.txt Parent Ref: 10469 Parent Seq: 1<br />
M: Thu Oct 2 23:22:05 2014 Z<br />
A: Thu Oct 2 23:22:05 2014 Z<br />
C: Thu Oct 2 23:22:05 2014 Z<br />
B: Thu Oct 2 23:22:05 2014 Z<br />
[RESIDENT]<i><br /></i><br />
<br />
Next, I used the program <a href="https://code.google.com/p/mft2csv/wiki/SetMACE">SetMACE</a> to change the $STANDARD_INFORMATION timestamps to "2010:07:29:03:30:45:789:1234" . As expected, the $STANDARD_INFORMATION changed, while the $FILE_NAME stayed the same. Once again, this is common to see in files that have been time stomped:<br />
<br />
12591 FILE Seq: 1 Link: 2 0x38 4 Flags: 1 <br />
[FILE]<br />
.\Documents and Settings\Mari\My Documents\New Text Document.txt<br />
<span style="color: red;"> M: Wed Jul 29 03:30:45 2010 Z<br /> A: Wed Jul 29 03:30:45 2010 Z<br /> C: Wed Jul 29 03:30:45 2010 Z<br /> B: Wed Jul 29 03:30:45 2010 Z</span><br />
FN: NEWTEX~1.TXT Parent Ref: 10469 Parent Seq: 1<br />
M: Thu Oct 2 23:22:05 2014 Z<br />
A: Thu Oct 2 23:22:05 2014 Z<br />
C: Thu Oct 2 23:22:05 2014 Z<br />
B: Thu Oct 2 23:22:05 2014 Z<br />
FN: New Text Document.txt Parent Ref: 10469 Parent Seq: 1<br />
M: Thu Oct 2 23:22:05 2014 Z<br />
A: Thu Oct 2 23:22:05 2014 Z<br />
C: Thu Oct 2 23:22:05 2014 Z<br />
B: Thu Oct 2 23:22:05 2014 Z<br />
<br />
Next, I used the rename command via the command prompt to rename the file from New Text Document.txt to "Renamed Text Document.txt" (I know - creative naming). The interesting thing here is, unlike the Windows 7 test where two dates were copied over, <i><b>all four dates</b></i> were copied over from the original files $STANDARD_INFORMATION into the $FILE_NAME:<br />
<br />
12591 FILE Seq: 1 Link: 2 0x38 6 Flags: 1 <br />
[FILE]<br />
.\Documents and Settings\Mari\My Documents\Renamed Text Document.txt<br />
M: Wed Jul 29 03:30:45 2010 Z<br />
A: Wed Jul 29 03:30:45 2010 Z<br />
C: Thu Oct 2 23:38:36 2014 Z<br />
B: Wed Jul 29 03:30:45 2010 Z<br />
FN: RENAME~1.TXT Parent Ref: 10469 Parent Seq: 1<br />
<span style="color: red;"> M: Wed Jul 29 03:30:45 2010 Z<br /> A: Wed Jul 29 03:30:45 2010 Z<br /> C: Wed Jul 29 03:30:45 2010 Z<br /> B: Wed Jul 29 03:30:45 2010 Z</span><br />
FN: Renamed Text Document.txt Parent Ref: 10469 Parent Seq: 1<br />
<span style="color: red;">M: Wed Jul 29 03:30:45 2010 Z<br /> A: Wed Jul 29 03:30:45 2010 Z<br /> C: Wed Jul 29 03:30:45 2010 Z<br /> B: Wed Jul 29 03:30:45 2010 Z</span><br />
<br />
Based upon my testing, a rename could have caused the 2010 dates to be the same in both the $SI and $FN attributes in my outliers. This scenario "in the wild" makes sense...the malware is dropped on the system, time stomped, then renamed to a file name that is less conspicuous on the system. This sequence of events on a Windows XP system may make it difficult to use the MFT analysis alone to identify time stomping.<br />
<br />
So what if you run across a file where you suspect this may be the case? On Windows XP you could check the restore points change.log files. These files track changes such as file creations and renames. Once again, Mr. HC has a perl script that parses these change log files, lscl.pl. If you see a file creation and a rename, you can use the restore point as a guideline to when the file was created and renamed on the system.<br />
<br />
You could also parse the USN change journal to see if and when the suspected file had been created and renamed. Tools such as <a href="https://www.gettriforce.com/product/triforce-anjp/">Triforce</a> or Harlan's usnj.pl do a great job.<br />
<br />
If the change.log file and and journal file do not go back far enough, checking the compile date of the suspicious file with program like <a href="http://www.ntcore.com/exsuite.php">CFF Explorer</a> may also help shed some light. If a program has a compile date years <b>after</b> the born date,.. red flag.<br />
<br />
I don't think anything I've found is ground breaking or new. In fact,the <a href="http://www.forensicswiki.org/wiki/Timestomp">Forensics Wiki</a> entry on timestomp demonstrates this behavior with time stomping and moved files, but I thought I would share anyways.<br />
<br />
Happy hunting, and may the odds be ever in your favor...<br />
<br />
<br />Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com1tag:blogger.com,1999:blog-2456910508235577340.post-54388097018059303232014-09-02T08:35:00.000-07:002014-09-02T08:35:17.010-07:00SQLite Deleted Data Parser - GUI AddedLast year I wrote a Python script to parse deleted data from SQLite Databases (<a href="http://az4n6.blogspot.com/2013/11/python-parser-to-recover-deleted-sqlite.html">original post here</a>).<br />
Every once in a while, I get emails asking for help on how to use the SQLite Parser from users who are not that familiar with using Python or command line tools in general.<br />
<br />
As an everyday user of command line tools and Python, I forget the little things that may challenge these users (we were all there at one point and time!) This includes things like quotes around file paths, which direction slashes go, and how to execute a python script if Python is not in your environment variable.<br />
<br />
So, to that end, I have created a Windows GUI for the SQLite Parser to make the process a little less painful.<br />
<br />
The GUI is pretty self explanatory:<br />
<ul>
<li>Choose the path to the SQLite database</li>
<li>Choose the file to save the results to</li>
<li>Select Formatted or Raw output</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxVa6PD79eUlHAyNMUmLgyaIQnh2079O0U8OWuo7gjDmzuUOkcPXAGNMJf0Bz-CRbbewhZGdm-QDB6GVelSVN3Dq47Ss8HJShnG7VMjFw4PDzhMFgYLYacFgOaJ6lQk-4WiH0KjDOy2uV8/s1600/SQLite+Deleted+Data+Parser.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxVa6PD79eUlHAyNMUmLgyaIQnh2079O0U8OWuo7gjDmzuUOkcPXAGNMJf0Bz-CRbbewhZGdm-QDB6GVelSVN3Dq47Ss8HJShnG7VMjFw4PDzhMFgYLYacFgOaJ6lQk-4WiH0KjDOy2uV8/s1600/SQLite+Deleted+Data+Parser.PNG" /></a></div>
<br />
This means there are now three flavors of the SQLParser available:<br />
<ul>
<li>sqlparse.py - python script</li>
<li>sqlparse_CLI.exe - Windows command line tool</li>
<li>sqlparse_GUI.exe - Windows GUI tool </li>
</ul>
All three files are available for download <a href="https://github.com/mdegrazia/SQLite-Deleted-Records-Parser/releases">here</a> on on my GitHub page.<br />
<br />
Coming soon... a blog post/tutorial on how to use python scripts :-)Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com1tag:blogger.com,1999:blog-2456910508235577340.post-11706221053608110202014-07-21T08:13:00.002-07:002014-07-21T10:17:45.734-07:00Safari and iPhone Internet History Parser<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilaquCfv9TTgyuCBTdwGEgZtMFVJEap2lkQxTRQxe2EzvLC7hbxoDAznwv6M71t-w8W3nXO624DdwmhXLytEijIz6OTODeKlp8mH_wgcFg3PFhf0G3jtnQ9f_I7fzd-23Z7pEllZe3_Awh/s1600/moomcomishs(1).png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEilaquCfv9TTgyuCBTdwGEgZtMFVJEap2lkQxTRQxe2EzvLC7hbxoDAznwv6M71t-w8W3nXO624DdwmhXLytEijIz6OTODeKlp8mH_wgcFg3PFhf0G3jtnQ9f_I7fzd-23Z7pEllZe3_Awh/s1600/moomcomishs(1).png" height="320" width="278" /></a>Back in June, I had the opportunity to speak at the <a href="http://www.sans.org/event/dfir-summit-2014" target="_blank">SANS DFIR Summit</a>. One of the great things about this conference was the ability to meet and socialize with all the attendees and presenters. While I was there, I had a chance to catch up with Sarah Edwards who teaches the Mac 518 class for SANS.<br />
<br />
I'm always looking for new projects to work on, and she suggested a script to parse Safari Internet History. So the 4th of July long weekend rolled around and I had some spare time to devote to a project. In between the fireworks and a couple of Netflix shows (OK, maybe 10 shows), I put together a python script that parses out several plist files related to Safari Internet History: History.plist, Bookmarks.plist, TopSites.plist and Downloads.plist.<br />
<br />
Since the iPhone also uses Safari, I decided to expand the script to
parse some iPhone Safari artifacts: History.plist, Bookmarks.db and RecentSearches.plist. I imagine the iPad also contains Safari Internet History, but I did not have one at my disposal to test. If you want to send one to me for testing, I would be happy to take it off your hands :-).<br />
<br />
In this post I'll run through each of the artifacts I located and explain how to use the script to parse out the files.<br />
<br />
<h3>
<b>Plist Files: A love/hate relationship</b></h3>
First, a little background on plist files. Plist files are awesome because they can contain all sorts of information such as Internet History, Recent Docs, Network IDs etc. There are free tools for both Windows and OS X that will allow you view the data stored in the plist file. For Windows, you can use <a href="http://www.icopybot.com/plist-editor.htm" target="_blank">plist Editor</a>. If you have a Mac, a free plist editor is included
in Apple's XCode Developer Tools which can be downloaded through the App
Store.<br />
<br />
However, plist files also stink because while the plist format is standardized, it's
entirely up to the programmer to store <i>whatever</i><b> </b>they want, in <i>whatever</i>
format they want.<br />
<br />
A (frustrating)
example of this is date information. In the Safari History.plist file
the date is defined as a "String", and is stored in Mac Absolute time. Mac
Absolute time is the number of seconds since <strike>June</strike> January 1, 2001. Below is an example of this from a Safari History.plist file viewed in the XCode plist editor:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMhgcoS-yDOtV4LCoiHIbQozvf0pxIIwowvkS4xXx7uX74cEe_lOABmi9NC9vvhJOfOZTIjDBNtUlKw56Guxr1FfFzD63RGqzqKsYTXjfUQOv6sktmXZ07olsVtc9vppmUztvXhALbaam9/s1600/history+date.png" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMhgcoS-yDOtV4LCoiHIbQozvf0pxIIwowvkS4xXx7uX74cEe_lOABmi9NC9vvhJOfOZTIjDBNtUlKw56Guxr1FfFzD63RGqzqKsYTXjfUQOv6sktmXZ07olsVtc9vppmUztvXhALbaam9/s1600/history+date.png" height="258" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>History.plist file in XCode plist editor</b></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
In
the Safari Bookmarks.plist file, the date is stored in a field defined
as "Date".The date is stored in a more standard format:</div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAW0oWkotaxSe4on5ONBQH2yiwnBj5UYAnyK-CEJ58FONZQank6-SrNavAwnrgW44iUdFZ7tah1A6VycgqSofby2LdYSz9hsRez-v56dpqrFXPh6cbZ8bEyyOpDaexN3JPJMbumRDnVGNA/s1600/bookmarks+date.png" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAW0oWkotaxSe4on5ONBQH2yiwnBj5UYAnyK-CEJ58FONZQank6-SrNavAwnrgW44iUdFZ7tah1A6VycgqSofby2LdYSz9hsRez-v56dpqrFXPh6cbZ8bEyyOpDaexN3JPJMbumRDnVGNA/s1600/bookmarks+date.png" height="220" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Bookmarks.plist file in XCode plist editor</b></td></tr>
</tbody></table>
This
means that each plist file needs to reviewed manually to determine what
format the data is in, and how it's stored before it can be parsed.<br />
<br />
So, moving on to the artifacts...<br />
<br />
<h3>
<b>Where's the beef?</b></h3>
On a Mac OS X, the Safari Internet History is located under the folder /Users/%USERNAME%/Library/Safari. As I mentioned before, I located four plist files in this folder containing Internet History: History.plist, Bookmarks.plist, TopSites.plist and Downloads.plist. I've written the script to read either an individual file, or the entire folder at once.<br />
<br />
(If you're wondering about the Safari cookie files, I already wrote a separate tool support these, which can be found on my <a href="http://az4n6.blogspot.com/p/downloads.html">downloads</a> page.)<br />
<br />
<b>History.plist</b><br />
This file contains the the last visited date, URL, page title and visit count. To run the parser over this file and get a tsv file use the following syntax:<br />
<br />
<div class="code">
safari_parser.py --history -f history.plist -o history-results.tsv </div>
<br />
<b>TopSites.plist</b><br />
The Top Site feature of Safari identifies 12 Top Sites based upon how often and how recent the sites were visited. There are several ways to view the tops sites in Safari, such as starting a new tab or selecting it from the Menu>View>Top Sites. Small thumbnails of each Top Site are displayed. The user has the option to Pin or Delete a site from the Top
Sites. Pinning a site keeps it in the Top Sites List, while deleting it
removes it. The list can be increased to hold up to 24 sites.<br />
<br />
The thumbnails for the webpage previews for Safari can be found under /Users/%Username%/Library/Caches/com.apple.Safari. Below is how the TopSites appear to a user ( this may vary depending on the browser version):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSmUlkgPkYVSbVJTO3oaBgbEjfKfYAe62cwnvYKSbsWKvLHdsgVvozw5BgveIPySBYBgAFXNq9g92STu_zON8b9M_wVPDDQHt9P_18na-1V6IBw3mPIPQoBgBT48UyJdh8kzy-ct_P-xmQ/s1600/Screen+Shot+2014-07-06+at+5.33.06+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSmUlkgPkYVSbVJTO3oaBgbEjfKfYAe62cwnvYKSbsWKvLHdsgVvozw5BgveIPySBYBgAFXNq9g92STu_zON8b9M_wVPDDQHt9P_18na-1V6IBw3mPIPQoBgBT48UyJdh8kzy-ct_P-xmQ/s1600/Screen+Shot+2014-07-06+at+5.33.06+PM.png" height="344" width="640" /></a></div>
<br />
<br />
The TopSite.plist file contains the Page Title and URL. It also stores values to indicate if it's a Pinned or Built in Site. Built in Sites are pre-populated sites such as iCloud or the Apple Website.<br />
<br />
TopSites that have been deleted are tracked in the TopSites.plist as "BannedURLStrings".<br />
<br />
To parse the TopSites.plist file use the following syntax:<br />
<br />
<div class="code">
safari_parser.py --topsites -f TopSites.plist -o topsite-results.tsv</div>
<br />
<b>Downloads.plist</b><br />
Downloads are stored in the Downloads.plist file. When a file is downloaded, an entry is made containing the following: 1)Download URL; 2)File name including the path where it was downloaded to; 3)Size of the file; 4)Number of bytes downloaded so far. The user may clear this list at anytime by selecting "Clear" from the Downloads dialog box:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEsbSAu61ChpLMZFEPYDYxL1yzzA0gMJEMidSWz-R48EXZ2lvrxBR28tcErmmjDl2mPnNzCA3ME2cAxnZyJnGdEL_VAw1uykXdWXfeRBxHC0b5Gdyk6wYrnLDO0RcZgw9AYYiSLhC-Wgxz/s1600/Screen+Shot+2014-07-06+at+5.33.40+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgEsbSAu61ChpLMZFEPYDYxL1yzzA0gMJEMidSWz-R48EXZ2lvrxBR28tcErmmjDl2mPnNzCA3ME2cAxnZyJnGdEL_VAw1uykXdWXfeRBxHC0b5Gdyk6wYrnLDO0RcZgw9AYYiSLhC-Wgxz/s1600/Screen+Shot+2014-07-06+at+5.33.40+PM.png" height="241" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
To parse the Downloads.plist file use the following syntax:<br />
<br />
<div class="code">
safari_parser.py --downloads -f Downloads.plist -o download-results.tsv</div>
<br />
<b>Bookmarks.plist</b><br />
Safari tracks three different types of bookmarks in the Bookmarks.plist file: Favorites, Bookmarks and the Reading List.<br />
<br />
<b>Favorites</b><br />
The Bookmarks Bar (aka Favorites) is located at the top of the browser:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiwwN37CYwBI0OxpQ4ihkxfRRqdNiGMAEn9Rouv5-uc7RaEYl0wWZlh6Rr0ZcgtGdox-BM1NMjsQetIWTBauw5wq5A6SDNaO0ANXMeJ7eJS42C8VHMetmq8H6kkFFY3dWa7t1snSzzP272/s1600/Bookmar+Bar.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiwwN37CYwBI0OxpQ4ihkxfRRqdNiGMAEn9Rouv5-uc7RaEYl0wWZlh6Rr0ZcgtGdox-BM1NMjsQetIWTBauw5wq5A6SDNaO0ANXMeJ7eJS42C8VHMetmq8H6kkFFY3dWa7t1snSzzP272/s1600/Bookmar+Bar.png" height="138" width="640" /></a></div>
<br />
The Favorites are also displayed on the side bar:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBSk10WkaNp4kAsUD-U_eHqcsWhPr4LMusReVft0KiYxIH0YkbsRB_nl9bAx_JgEUoDuPixush0ot2DmaGSuj9JGeFogwBn-jwrAsR_xeftntU8dORVtFaGnvxOMuf2B5Ym1B8wZucrU2j/s1600/Safari+Favorites.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBSk10WkaNp4kAsUD-U_eHqcsWhPr4LMusReVft0KiYxIH0YkbsRB_nl9bAx_JgEUoDuPixush0ot2DmaGSuj9JGeFogwBn-jwrAsR_xeftntU8dORVtFaGnvxOMuf2B5Ym1B8wZucrU2j/s1600/Safari+Favorites.png" height="640" width="395" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<b>Bookmark Menu</b><br />
A folder titled "Bookmark Menu" is created by default when a user creates bookmarks. It contains a hierarchical structure of bookmarks and folders - these are shown in the red box below:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHKEtnEpLter3WFUYbMESDuKBf-VS1Boq-UF_us-PqtyOqDdW2Rq73M9zyKHu2-XoQMDr5TpqzxB3cM5-RqPdvL9sSkuEvFLIgzgN3VBnSVNR_c_6D3XW3llFBcgZfWCiXJdHO-o73gV6Q/s1600/Bookmark+Menu.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHKEtnEpLter3WFUYbMESDuKBf-VS1Boq-UF_us-PqtyOqDdW2Rq73M9zyKHu2-XoQMDr5TpqzxB3cM5-RqPdvL9sSkuEvFLIgzgN3VBnSVNR_c_6D3XW3llFBcgZfWCiXJdHO-o73gV6Q/s1600/Bookmark+Menu.png" height="640" width="396" /></a></div>
<br />
The user may add folders, as demonstrated with the "test bookmarks" folder below:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAC1wDJkKg4ynhRHOjoVi4s3NNFgvh02Vs5cXwsVEMATvuclDr95uKZcE9_7-Pyruavky_T83AsFA8yeeE72W8TMpzuiT_dTAOBU8zNXBLL-CfFt9pqxS77gxQVrpxxOH-x4nyFz2thWhe/s1600/user-added.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAC1wDJkKg4ynhRHOjoVi4s3NNFgvh02Vs5cXwsVEMATvuclDr95uKZcE9_7-Pyruavky_T83AsFA8yeeE72W8TMpzuiT_dTAOBU8zNXBLL-CfFt9pqxS77gxQVrpxxOH-x4nyFz2thWhe/s1600/user-added.png" height="640" width="396" /></a></div>
<br />
<b>Reading List</b><br />
The Reading List is another type of bookmark. According to Safari documentation, "Reading List helps you save webpages and links for you to read later, even when you are not connected to the internet". These items show up when the user selects the Reading List icon:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihks6p5Mqe8BqtKZ5g2WES6_FKUHb_3eIlm9URtnnTBHX4-S_h7TY_ubeDGEon1aQASknQU8GsEdH5wLNVIZefXC54ABhVn5wLXTD771aqMSYb1stOpUJjrOcoGO7jd-h1kbmx8BpOMFgw/s1600/Reading+List.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihks6p5Mqe8BqtKZ5g2WES6_FKUHb_3eIlm9URtnnTBHX4-S_h7TY_ubeDGEon1aQASknQU8GsEdH5wLNVIZefXC54ABhVn5wLXTD771aqMSYb1stOpUJjrOcoGO7jd-h1kbmx8BpOMFgw/s1600/Reading+List.png" height="640" width="436" /></a></div>
<br />
Safari downloads and stores information such as cached pages related to the Reading List under /Users/%USERNAME%/Library/Safari/ReadingListArchives. I didn't spend too much time researching this as my parser is focused on the bookmarks.plist file, but keep it in mind as it may turn up some interesting stuff.<br />
<br />
All three types of bookmarks (Favorites, Bookmarks and Reading Lists) are stored in the Bookmarks.plist file.<br />
<br />
The Bookmarks.plist file tracks the Page Title and URL for the Favorites and the Bookmarks, however, the Reading List entries contain a little bit more information. The Reading Lists also contains a date added, date last fetched, fetch result, and preview text. There are also a couple of boolean entries, Added Locally and Archived on Disk.<br />
<br />
Out of all the plist files mentioned so far, I think this one looks the most confusing in the plist editor programs. The parent/child relationships of the folders and sub folders can get pretty messy:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwtHVVMwMqv4cuIPFUcnCmf3e8GSwVSYjX8b6aco8YxiUoR4P5hzSwiTk0TUKKDA9wnd9ps-A9zYfrSsMvjh3B3IaBgAAyUX6MKngg3yAbSY8ZW47acTYaLJJtoJ6jtIqO8DD0x3r916HF/s1600/bookmarks.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwtHVVMwMqv4cuIPFUcnCmf3e8GSwVSYjX8b6aco8YxiUoR4P5hzSwiTk0TUKKDA9wnd9ps-A9zYfrSsMvjh3B3IaBgAAyUX6MKngg3yAbSY8ZW47acTYaLJJtoJ6jtIqO8DD0x3r916HF/s1600/bookmarks.PNG" height="400" width="366" /></a></div>
<br />
To parse the Bookmarks.plist file, use the following syntax:<br />
<br />
<div class="code">
safari_parser.py --bookmarks -f Bookmarks.plist -o bookmark-results.tsv</div>
<br />
The Safari Parser will output this into a spreadsheet with the folder structure rebuilt, which is hopefully more intuitive then viewing in the plist editor: <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGmjHyfEkvSt_MWlSEulHx2ivzqf4to_RewVRPhNg18gISgLdhRwED1PEj4F5LYkeVEGXurbEASCkOYCDuyy4INBgR4-3z3zUHD1gLrhI2AMvIq-xhIES7M7xrMhKNz-70CcG3jV-0bwND/s1600/bookmark+tsv.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGmjHyfEkvSt_MWlSEulHx2ivzqf4to_RewVRPhNg18gISgLdhRwED1PEj4F5LYkeVEGXurbEASCkOYCDuyy4INBgR4-3z3zUHD1gLrhI2AMvIq-xhIES7M7xrMhKNz-70CcG3jV-0bwND/s1600/bookmark+tsv.PNG" height="240" width="640" /></a></div>
<br />
<br />
<b>All Four One and One for All</b><br />
Instead of parsing each file individually, all four files can be parsed by pointing Safari Parser to a folder containing all four files. This means you can export out the /Users/%Username%/Library/Safari folder and point the script at it. You could also mount the image and point it to the mounted folder. To parse the folder, use the following syntax:<br />
<br />
<div class="code">
safari_parser.py -d /Users/maridegrazia/Library/Safari -o /Cases/InternetHistory/Reports</div>
<br />
This will create four tsv files with results from each of the above Internet History Files.<br />
<b><br /></b>
<br />
<h3>
<b>iPhone Internet History</b></h3>
Safari is also installed on the iPhone so I figured while I was at it I might as well expand the script to handle the iPhone Internet History files. I had some test data laying around, and I was able to locate three files of interest: History.plist, Bookmarks.db and RecentSearches.plist.<br />
<br />
While my test data came from an iPhone extraction, these types of files are also located in an iTunes backup on a computer. This means that even if you don't have access to the phone, you could get still get the Internet History. Check in the user's folder under \AppData\Roaming\Apple Computer\MobileSync\Backup, then use a tool like<a href="https://code.google.com/p/iphonebackupbrowser/"> iphonebackupbrowser</a> to browse the backups and export out the files:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgf1gedwfdCY3bafQqHy0ZP8TQF4I23qi56-gMk3P0wDbdN8g-hZBYrE9bIy2lmODYPWNinLHKgMP3B2-LRrUNxOqbTdNtN1XMmFnRqXp8aMuBZG3SYY_65BkvMqhbIhmG-Iu7lM67ja2k0/s1600/iphone-backup-2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgf1gedwfdCY3bafQqHy0ZP8TQF4I23qi56-gMk3P0wDbdN8g-hZBYrE9bIy2lmODYPWNinLHKgMP3B2-LRrUNxOqbTdNtN1XMmFnRqXp8aMuBZG3SYY_65BkvMqhbIhmG-Iu7lM67ja2k0/s1600/iphone-backup-2.PNG" height="396" width="400" /></a></div>
<br />
<b>History</b><br />
The location of the History.plist file may vary depending on the model of the iPhone. Check \private\var\mobile\Library\Safari or \data\mobile\Library\Safari for this file.<br />
<br />
Luckily, the History.plist file has the same format as the OS X version, so using the script to parse the iPhone History.plist file works the same:<br />
<br />
<div class="code">
safari_parser.py --history -f history.plist -o history-results.tsv</div>
<br />
<b>Bookmarks</b><br />
The location of the Bookmarks.db file may vary depending on the model of the iPhone. Check \private\var\mobile\Library\Safari or \data\mobile\Library\Safari for this file.<span style="color: red;"> </span>On an iPhone, this file is stored in an SQLite database rather then the plist format used on OS X. In the test data I had, I did not see any entries for the Reading List. To parse the iPhone Bookmarks.db file, use the following syntax:<br />
<br />
<div class="code">
safari_parser.py --iPhonebookmarks -f bookmarks.db -o bookmark-results.tsv</div>
<b><br /></b>
<b>Recent Searches</b><br />
I located a RecentSearches.plist file under the cache folder. The location of this file may vary depending on the model of the iPhone. Check \private\var\mobile\Library\Caches\Safari or \data\mobile\Library\Caches\Safari. This file contained a list of recent searches, about 20 or so. Use the following syntax to parse this file:<br />
<br />
<div class="code">
safari_parser.py --iPhonerecentsearches -f recentsearches.plist -o recentsearches-results.tsv</div>
<br />
You can also point the script to a directory with all three files and parse them at once:<br />
<br />
<div class="code">
safari_parser.py -d /Users/maridegrazia/iPhoneFiles -o /Cases/InternetHistory/Reports</div>
<br />
<h3>
The Script</h3>
The Safari Parser can be download <a href="https://github.com/mdegrazia/Safari-Internet-History-Parser" target="_blank">here</a>. It requires the biplist library which is super easy to install (directions below). However, I've also included a complied .exe file for Windows if you don't want to hassle with installing the library. A thank you to <a href="http://windowsir.blogspot.com/" target="_blank">Harlan Carvey</a> for suggesting the <a href="http://www.pyinstaller.org/" target="_blank">PyInstaller </a>to compile Windows binaries for python - it worked like a charm.<br />
<br />
To install biplist in Linux just type the following:<br />
<br />
<div class="code">
sudo easy_install biplist
</div>
<br />
For Windows, if you don't already have it installed, you'll need to grab the easy install utility which is included in the <a href="https://pypi.python.org/pypi/setuptools">setup tools</a>
from python.org. The setup tools will place easy_install.exe into your Python directory in the Scripts folder. Change into this directory and run:<br />
<br />
<div class="code">
easy_install.exe biplist</div>
<br />
Remember to look at the plist files to manually to verify your results. I don't have access to every past or future version of Safari or iOS. As always, just shoot me an email or tweet if you need some modifications made.<br />
<br />
<h3>
References and Tools</h3>
<div id="main-title">
<a href="https://github.com/mdegrazia/Safari-Internet-History-Parser" target="_blank">safari_parser.py</a> (my script to parse the Safari Internet History)<br />
<a href="http://support.apple.com/kb/PH5059">Safari 5.1 (OS X Lion): View and customize Top Sites</a> </div>
<div id="main-title">
<a href="http://support.apple.com/kb/PH5074?viewlocale=en_US">Safari 5.1 (OS X Lion): Save articles to read later with Reading List</a></div>
<div id="main-title">
<a href="http://www.icopybot.com/plist-editor.htm">Plist Editor</a> (free plist editor for Windows)</div>
<div id="main-title">
<a href="https://developer.apple.com/xcode/">XCode </a>(includes free Plist Editor for OS X) <br />
<a href="https://code.google.com/p/iphonebackupbrowser/">iphonebackupbrowser</a> ( free iTunes backup browser)<br />
<br /></div>
Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com5tag:blogger.com,1999:blog-2456910508235577340.post-2387064151068593462014-04-24T08:14:00.002-07:002015-09-15T21:10:26.965-07:00What's the Word - Thunderbird! - Parser that is....Thunderbird is a free email client by Mozilla (similar to Outlook). Most of the major Forensic tools support parsing this data in one way or another. However, I recently came across a Thunderbird profile in a Volume Shadow Copy that was not getting parsed correctly, or in some instances, into a format that I needed it in. <br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: right; margin-left: 1em; text-align: right;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9jWgsDjYSMA66CrariE-t1fw3xTr-ex6wavH3nTeh1zEq9iDDa5zxM_DH2EnrGqFUrfqq9vmN1N5FH_oqPHGy3btO4F0FmCNl5QcLK7GzPA_LpiJzYxaPhIg06jtP9-iuMimOpLjI8Zid/s1600/email-snake.jpg" imageanchor="1" style="clear: right; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9jWgsDjYSMA66CrariE-t1fw3xTr-ex6wavH3nTeh1zEq9iDDa5zxM_DH2EnrGqFUrfqq9vmN1N5FH_oqPHGy3btO4F0FmCNl5QcLK7GzPA_LpiJzYxaPhIg06jtP9-iuMimOpLjI8Zid/s1600/email-snake.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
<br />
What tipped me off that the profile was not being parsed correctly? Several things. One program I used parsed only 274 messages. Based upon the large size of the profile, this seemed suspect to me. I tried another program and it parsed over 5,000 emails from the same profile. Quite a discrepancy. When I tried to view the profile natively using Thunderbird, it threw errors.<br />
<br />
This caused me to take a closer look at the Thunderbird files, and
untimely, write a python parser to extract the
emails – including deleted ones. <br />
<b> </b><br />
<b>Testing </b><br />
<br />
Because the email profile was corrupted, I wanted to test the same programs with a "normal" profile. I actually use Thunderbird as my email client, so I had a decent profile for testing with over 7,000 emails in my Inbox and about 3,300 in my sent folder over the course of a couple of years.<br />
<br />
<br />
I parsed my profile with three forensic programs as well as just viewing it in Thunderbird. I also ran the python script I wrote over it (noted as TB Parser below). I was surprised by the variety of results - many programs were not getting all the messages. I've listed the major email folders from the Thunderbird profile below and the number of parsed emails from each program:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVou2JhbF-8DTK6WgYei66DCE2Z8xHrUnTPoUc5YxVd4QSMIzEIYjdUmSth6yaXo31hq7yfWFc14BNWKpkSOBi9lcVs-g__k69X0xVYKXkW1ye_romrqsCJKwkJL9L88_LEteMilPbmzUN/s1600/table.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVou2JhbF-8DTK6WgYei66DCE2Z8xHrUnTPoUc5YxVd4QSMIzEIYjdUmSth6yaXo31hq7yfWFc14BNWKpkSOBi9lcVs-g__k69X0xVYKXkW1ye_romrqsCJKwkJL9L88_LEteMilPbmzUN/s1600/table.PNG" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Tool 1 is a common "all in one" forensic tool. If you look at results from the Inbox, over 4,000 emails were parsed. If an examiner was using this as their only tool, it's easy to see how they might not even realize that an additional 3,000 messages were not parsed. <br />
<br />
A possible reason for these discrepancy is the format in which Thunderbird stores its emails. Thunderbird uses a modified version of the <a href="http://www.digitalpreservation.gov/formats/fdd/fdd000383.shtml">MBOX</a> email format, called <a href="http://www.digitalpreservation.gov/formats/fdd/fdd000385.shtml#identification">MBOXRD</a><sup>1</sup>.This may account for the partial processing of emails as many of the tools support state support for MBOX. However, Tool 1 states in it's documentation specific support for Thunderbird.<br />
<br />
So if the tools states support for Thunderbird, or if you see some emails but they are <b>all</b> not being parsed, is the tool to blame? I think it may be a little misleading that some of the emails are parsed, however, I believe that it is incumbent upon the examiner to verify the results and understand the way that the tools work. However, that being said, sometimes it's easier said then done. I had a situation where it was pretty obvious all the emails had not been parsed. What if the profile size was 1GB and 5,000 emails were parsed? Is that a reasonable number? What if it was supposed to be 6,000 and your smoking gun is one on the ones that didn't get parsed?<br />
<br />
<br />
<b>Thunderbird Configuration</b><br />
<br />
First, a little background information on Thunderbird. Thunderbird allows a user to set up both POP and IMAP email. Once a user has set up and configured their profile, it’s stored under the following location (at least on Windows 7):<br />
<br />
C:\Users\%USERNAME%\AppData\Roaming\Thunderbird\Profiles\[Random].default<br />
<br />
Unlike Outlook, the data is not stored in one file, but rather a series of files and folders under the profile directory. If you want to view this profile natively with Thunderbird, the easiest way I have found so far it to launch Thunderbird from the command prompt with the –profile switch and point it to the path where you have exported out the profile. Make sure you’re not connected to the Internet if your doing this on an evidence profile. The last thing you want to do is download new email or send out a message that has been sitting in the outbox. This may (and probably will) modify the file, so only do it on a copy.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMn-kCPEhENnP2dOdoV_emgmJLK4_PAGRKG4uNi2s5G02PyAIJOAyqlpFtThl2SwSKkjQJ8kiV5zcAwdZHopNo8ksp28ziaXUcm9hVDYF4iHEhjjJp6aeUKvrWS50jOjM26kQ7L48SEgdG/s1600/thunderbird+profile+launch.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMn-kCPEhENnP2dOdoV_emgmJLK4_PAGRKG4uNi2s5G02PyAIJOAyqlpFtThl2SwSKkjQJ8kiV5zcAwdZHopNo8ksp28ziaXUcm9hVDYF4iHEhjjJp6aeUKvrWS50jOjM26kQ7L48SEgdG/s1600/thunderbird+profile+launch.PNG" /></a></div>
<div>
</div>
<div>
<br /></div>
<div>
Once launched, a typical setup may look like this:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid3QfsG9l1-f8lgNyGObSVLmnesrzAjEu00YORi7Ari9wmqy1NQyt1eYy49ApWnNCbe7kg05vAs7Pot6ExvbsPvlZwycCElMEOPkHZRr61qg5cHctxIX8tUA9b8EbdqTB6hOeMj6mQreZq/s1600/typical-thunderbrid.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="518" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid3QfsG9l1-f8lgNyGObSVLmnesrzAjEu00YORi7Ari9wmqy1NQyt1eYy49ApWnNCbe7kg05vAs7Pot6ExvbsPvlZwycCElMEOPkHZRr61qg5cHctxIX8tUA9b8EbdqTB6hOeMj6mQreZq/s1600/typical-thunderbrid.png" width="640" /></a></div>
<div>
<br /></div>
<div>
Of course, being forensicators, this may not be the preferred way to review emails - but sometimes it's nice or even necessary to see files in the native viewer/program.</div>
<div>
<br /></div>
<div>
A whole bunch of files are created under the root of the profile directory. These include files like cookies.sqlite, places.sqite and formhistory.sqlite that may warrant a peek. However, I am going to focus on the email files for now.</div>
<div>
<br />
<b>Email Files</b></div>
<div>
<br />
Thunderbird stores the IMAP mail profile in a sub folder named "ImapMail" while POP mail and Local Folders are stored in a sub folder named "Mail":</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgcfZmjB16CuI6XvxZb7i0uu9ERRLIdlopFkFm3DHX8ZygbFn0xgN9CMp6OG5_xyVDJD1XecRL6Ab2A2t0eey9fdXGyd9HFDspfvt99ohD_OWFejO_7Q6-3IXahzBrZ3su3ieSIHGtege1_/s1600/imap+and+pop+email.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgcfZmjB16CuI6XvxZb7i0uu9ERRLIdlopFkFm3DHX8ZygbFn0xgN9CMp6OG5_xyVDJD1XecRL6Ab2A2t0eey9fdXGyd9HFDspfvt99ohD_OWFejO_7Q6-3IXahzBrZ3su3ieSIHGtege1_/s1600/imap+and+pop+email.png" width="640" /></a></div>
<div>
<br /></div>
<div>
</div>
<div>
</div>
<div>
</div>
<div>
</div>
<div>
<br />
There are several files that hold information related to emails. The first is the global-messages-db.sqlite file. This file is located in the root of the profile folder:</div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0NAFL-ed2IaxrZ8AlaRsgoZJuH1wB5vj6fTYiQ0G63KkEjQxMmJM51PdiE-p6aTcC9SLptOant4kK4hzvAEVFvjrsrmH_m2L-iknTcO2zwEnsX-qveZ-fdtCidZIm2DrLB9rIxVMjs-9A/s1600/global-messages-db.sqlite.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="443" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0NAFL-ed2IaxrZ8AlaRsgoZJuH1wB5vj6fTYiQ0G63KkEjQxMmJM51PdiE-p6aTcC9SLptOant4kK4hzvAEVFvjrsrmH_m2L-iknTcO2zwEnsX-qveZ-fdtCidZIm2DrLB9rIxVMjs-9A/s1600/global-messages-db.sqlite.png" width="640" /></a></div>
</div>
<div>
<br /></div>
<div>
<b>Global-messages.db.sqlite Database</b><br />
<b><br /></b>
The global-messages.db.sqlite is an SQLite database that Thunderbird uses to index and search messages.<sup>2</sup> This file can be viewed using an <a href="http://sourceforge.net/projects/sqlitebrowser/">SQLite Browser</a>. The "mesagesText_Contents" table contains the Email Body, Subject, Author, Recipients and Attachment Names. <br />
<div style="text-align: justify;">
<b></b></div>
</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td class="tr-caption" style="text-align: center;"><b>messagesText_Contents Table</b></td></tr>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYpQlBAE_CkLsCYuggAfVbquqeeNUsvHpL3B8Pb9tKV0eaC7huvLO310u0FFDnEw1bM3qk_eODvb_GUi2pNiHutRKlSXxZKCOufclJ5rbCJOg4ohcu3suCg5R3brr0H-fTB_fmS5S5dXqB/s1600/sqliteDB-sanatized.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="169" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYpQlBAE_CkLsCYuggAfVbquqeeNUsvHpL3B8Pb9tKV0eaC7huvLO310u0FFDnEw1bM3qk_eODvb_GUi2pNiHutRKlSXxZKCOufclJ5rbCJOg4ohcu3suCg5R3brr0H-fTB_fmS5S5dXqB/s1600/sqliteDB-sanatized.png" width="640" /></a></td></tr>
</tbody></table>
<div>
<div>
While this database contains email information, the email body is not a
true representation of the email. For example, the body field does not
contain images or attachments. Also, it does not contain messages that
have been deleted, whereas the MBOXRD file can (discussed below).
However, it does contain some useful data, such as the name of the
attachments of non-deleted emails. You could browse this database quickly to see if any
attachment names are suspicious. <br />
<br />
Using "docid" in the messagesText_Contents table, you can link it back to the “messages”table id field. The messages table contains information about each message, such as the headerMessageID and jsonAttributes. The jsonAttirbutes are what stores whether a message has been read, forwarded or replied to among other things.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8Hcbs7fTTwTykadU9z0z1Hc9HJtU2K_xbCizcIlHOlgcPNRzv23es1Sy08W9eEEheNnF9e3bq5R9nqNEwYYK4ACAeB4Q7r-b8xax0EaCgNFon1ZK7du-kwco3TBF8KPT3lEKKs1WhCFgj/s1600/thunderbird+messages+table.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="212" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8Hcbs7fTTwTykadU9z0z1Hc9HJtU2K_xbCizcIlHOlgcPNRzv23es1Sy08W9eEEheNnF9e3bq5R9nqNEwYYK4ACAeB4Q7r-b8xax0EaCgNFon1ZK7du-kwco3TBF8KPT3lEKKs1WhCFgj/s1600/thunderbird+messages+table.png" width="640" /></a></div>
<br />
The headerMessageID is also located in the MBOXRD file - which is what I used to link the raw MBOXRD data back to global-messages.db.sqlite database. You may noticed there is a deleted column here. Based upon limited testing, I believe that this value is used during the synching of the IMAP mail. When a message is deleted, it remains in this database with a 1 until the corresponding message is deleted on the mail sever. Once it has been deleted, the message is removed from the database, but remains in the MBOXRD file. Normally all these values will be '0' unless the user was offline when the message was deleted. <br />
<br />
<br />
In my particular case, this sqlite file was corrupt and I did not have
access to these tables. This may also be why one of the programs did
not parse the email fully - maybe it was relying on the table, who
knows. I have written my parser so that it does not need this database
to process the emails. It merely displays "Data not available" for the
fields that it can't pull from the table.</div>
<div>
<br /></div>
</div>
<div>
</div>
<div>
Just a heads up, there is more data that could be mined from this database, such as IM Conversations but I am trying to stay focused on email.. so moving on.... (and who uses Thunderbird to IM anyways????)</div>
<div>
<br />
<b>MBOXRD aka The Payload</b><br />
<b><br /></b></div>
<div>
Thunderbird stores email in an mbox format called <a href="http://www.digitalpreservation.gov/formats/fdd/fdd000385.shtml#identification">MBOXRD</a>. Basically, it stores email in plain text <a href="http://en.wikipedia.org/wiki/MIME">MIME </a>format. The cool thing is (based upon my testing and some <a href="http://kb.mozillazine.org/Compacting_folders">internet research</a>) when an email is deleted, it stays in this file. These deleted emails would not be seen if this profile was viewed using the Thunderbird client. The thunderbird parser pulls all the emails from these files, including deleted ones. </div>
<div>
<br /></div>
<div>
The MBOXRD files are stored in file that is named after the corresponding email folder with no file extension. For example, the Inbox folder stores its emails in the "INBOX" file":</div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjO5ac7Rt8RP8A7JypiBcxWiKK-JR6AjM7gBBzA8d1eIf7yDDMRBpy4QXFJ0n3EwbTGwEGZ3OmigKxN2HatYpDDPEFZuvuCgshpQGsnZv1y9-w__yLLPVJQuotXaLttHdmYYx4qgC5bcT6X/s1600/inbox.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="368" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjO5ac7Rt8RP8A7JypiBcxWiKK-JR6AjM7gBBzA8d1eIf7yDDMRBpy4QXFJ0n3EwbTGwEGZ3OmigKxN2HatYpDDPEFZuvuCgshpQGsnZv1y9-w__yLLPVJQuotXaLttHdmYYx4qgC5bcT6X/s1600/inbox.png" width="640" /></a></div>
</div>
<div>
<br /></div>
<div>
One level deeper, in the .sdb folder are the other folders such as the Sent folder and any user created folders to store email:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqMeWU8Nw4Jr1Caw-dm860jIeZUf7MdhF32KdAUfO7s6CJSf8yWxG_BSxoIFpIbb1UShT0KAYdRoZrJc5kgD_KYImXUHsjXf2UGywL5b0M-5ptE3pSRI_xVln8qLfwKH861-U2fQCJ7gZu/s1600/other-folders.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="378" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqMeWU8Nw4Jr1Caw-dm860jIeZUf7MdhF32KdAUfO7s6CJSf8yWxG_BSxoIFpIbb1UShT0KAYdRoZrJc5kgD_KYImXUHsjXf2UGywL5b0M-5ptE3pSRI_xVln8qLfwKH861-U2fQCJ7gZu/s1600/other-folders.png" width="640" /></a></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
.<b>MSF files</b><br />
For each MBOXRD file, there is a corresponding .msf file. The .msf file contains folder indexes and preference data in Mork format. According to internet research, this file format has taken a lot of heat as being a pain to work with. The pointers for messages marked as Junk by Thunderbird appear to be tracked in here (based upon my limited testing). However, the formatting of the Message-ID's in this file are whacked. They include backslashes and if they are to long, they can also include the newline "\n" character as well.<br />
<br />
<b>Deleted Files</b><br />
As mentioned before, when a file is deleted it is removed from the database yet still remains in the MBOXRD file. In order to determine if a file is deleted, the
headerMessageID in the MBOXRD file can be cross referenced back to the database. However, emails that have been marked as "Junk" mail by Thunderbird are not stored in the global-messages.db.sqlite either. The "Junk" emails appear to be stored in the corresponding MBOXRD .msf file. So two checks need to be done to determine if a file has been deleted. The logic is as follows:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsvsTugWXXxHIll2Bab8DdxRFbmdjghf7-KB3g1tUeb9xaf7J5Dl_7ezuJQSNyGA-EGb_fqmATIQMcMN6rkFCCnQvmky1IViwWtbrfTofmRrq5Dce4BKZXwAVCKsstKmkgVc5XLVqJE351/s1600/email-message-flowchart.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsvsTugWXXxHIll2Bab8DdxRFbmdjghf7-KB3g1tUeb9xaf7J5Dl_7ezuJQSNyGA-EGb_fqmATIQMcMN6rkFCCnQvmky1IViwWtbrfTofmRrq5Dce4BKZXwAVCKsstKmkgVc5XLVqJE351/s1600/email-message-flowchart.PNG" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjbYdZhecHxoC9dDukE71T5UIeP2foRDQSHpoCC1EGykpvhwcgWsyVfYiRwh46BS7FR5abur2bszl2KqPT734BQuxpLPucHnyYCXjIO4uODNiy3svLg4-xeOPA_NrstYkPRt6N0w8A6CmdJ/s1600/flowchart.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><br /></a></div>
</div>
<div>
<div>
<br />
<b>Thunderbird Email Parser</b></div>
<div>
<br /></div>
<div>
The python thunderbird email parser does three things:<br />
<br />
1) Provides an Excel Sheet with the following information: file the email came from, address information (from, to, cc, bc), subject, raw date, converted date (in UTC) a link to the exported email and a list of attachments:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIRt1geeLBv_L5TEvt6Y21iijvIDG9nVKiwC3H3H0it-jDIfn5g5ImjFtKAaXrdfcK3km6VgYJ6OZ-J0nM1014z_a5v0DNfgBCTdkx7csZQRvbirIyaATpb-BsYmACLbGS5netTFHqKShE/s1600/report.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="124" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIRt1geeLBv_L5TEvt6Y21iijvIDG9nVKiwC3H3H0it-jDIfn5g5ImjFtKAaXrdfcK3km6VgYJ6OZ-J0nM1014z_a5v0DNfgBCTdkx7csZQRvbirIyaATpb-BsYmACLbGS5netTFHqKShE/s1600/report.PNG" width="640" /></a></div>
<br />
<br />
2) If the corresponding global-messages.db.sqlite is readable, it will provide TRUE/FALSE values for read, replied forwarded and if the message was deleted. If a message was deleted, the database format has changed, or the database is corrupt, these fields will say "Data not available". <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFt0HNpGq2RYpiy_FhwJoXACHazQz-dFHPtMhyphenhyphenMGeUbLG60OuHGHQMBk8zS1NTswUsFWMPYaAursfELqdFVgi5Wt3rJj4pwYfb3XQda44Q806_ZzKXJ9gaWqP72LucBbfpREUh7L_E5wIM/s1600/db-fields.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFt0HNpGq2RYpiy_FhwJoXACHazQz-dFHPtMhyphenhyphenMGeUbLG60OuHGHQMBk8zS1NTswUsFWMPYaAursfELqdFVgi5Wt3rJj4pwYfb3XQda44Q806_ZzKXJ9gaWqP72LucBbfpREUh7L_E5wIM/s1600/db-fields.PNG" width="640" /></a></div>
<br />
<br />
<br />
3) It exports all the emails into a subfolder named "emails". Each email is named with the timestamp, email subject and unique number.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOAxWWBuTNLCEXW7opc3EMtVeLL5nXEnqrDQ816jrZyXnJQCUQcrygB3YVdHJKL8D_dD3xI_ypOdXGois0ye1RHH9m7fvz3JneGdaN8J9WB1bmUr58ikVmOWJ81znGia7pz4uWU4lkmS7-/s1600/thunderbird-exported-emails.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="378" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOAxWWBuTNLCEXW7opc3EMtVeLL5nXEnqrDQ816jrZyXnJQCUQcrygB3YVdHJKL8D_dD3xI_ypOdXGois0ye1RHH9m7fvz3JneGdaN8J9WB1bmUr58ikVmOWJ81znGia7pz4uWU4lkmS7-/s1600/thunderbird-exported-emails.PNG" width="640" /></a></div>
<br /></div>
<div>
</div>
<div>
<br />
Normally, when I write a parser, I like to dump the output into a CSV, TSV or a plain text file. This proved difficult for two main reasons. </div>
<div>
<br /></div>
<div>
First, many of the email addresses and strings within the email body contained tabs and commas which threw the formatting off.</div>
<div>
<br /></div>
<div>
Second, I needed a way to supply the body of the email. Putting a large body of an email into one cell looked ugly. Also, html was not displayed as one would see it in an email client making it difficult to read.<br />
<br />
For this reason, I decided to put the output into an Excel sheet. So in order to use the parser, the xlwt python libary needs to be installed which is pretty quick and easy to do for either the Windows or Linux platform. For Linux, you can use easy install. For Windows, you can downalod the installer for xlwt at <a href="https://pypi.python.org/pypi/xlwt/0.7.2">https://pypi.python.org/pypi/xlwt/0.7.2</a></div>
<div>
<br /></div>
<div>
To use the parser, simply point it at the profile directory and select a directory for the output. The script will recurse through all subdirectories, so if you export out the user profile, make sure it goes in it’s own directory:<br />
<br /></div>
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiphSWIOvj38xdC_h5UDNfenG3ba13-DjolbBVhhG5Ws19oBQ0ucXl7ocMM-PRr6VuTa0UiaOlIDKrkkbAP4TUs151QHraU4054H64AZLxYamkgw0UhQp-c0Awv6RztjvU4AMm20S8k2V5Q/s1600/thunderbird-parser-example.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiphSWIOvj38xdC_h5UDNfenG3ba13-DjolbBVhhG5Ws19oBQ0ucXl7ocMM-PRr6VuTa0UiaOlIDKrkkbAP4TUs151QHraU4054H64AZLxYamkgw0UhQp-c0Awv6RztjvU4AMm20S8k2V5Q/s1600/thunderbird-parser-example.PNG" /></a></div>
<br /></div>
<div>
</div>
<div>
A report.xls file will be created along with a log file in output folder. The .eml files will be placed in a subdirectory named “emails”.</div>
<div>
<br />
Some things to note, you may notice duplicate emails. This is because some emails may be stored in several folders, thus the email is stored in multiple
files. For example, an email may be in the Inbox, as well as the All
Email folder. Why not remove duplicate emails? Well, there may be significance if you find an email has been stored in a particular folder.<br />
<br />
I am using a built in MIME python library
to parse the emails. If an email does not follow this standard, the
output may not be as expected -weird characters, etc. This is why I put the file name in the Excel sheet. You can always refer back to the original MBOXRD file to verify the results.<br />
<br />
Although I have made every effort to test this script, and to make sure it is working accurately, verify your own results - which you should be doing anyways, right? ;-)<br />
<br />
For deleted emails, I have made the notation "Deleted (Verify)". I did this because there is not a specific flag or variable to designate that the email has been deleted. I run through several checks to located the Message-ID to determine if the file has been deleted. It seems to be working pretty good, but I have a limited set of test data. How can you verify if the message has been deleted? One way would be to open the profile in Thunderbird and use Thunderbird to search for the email. If the user deleted the email, it would not show up in Thunderbird.<br />
<br />
I have tested this on Thunderbird 24.4.0 using Windows 7 and the SIFT workstation with Python 2.7. If you want a Python 3+ version, I like shiny things and K-cup hot chocolate. <br />
<br />
Given the frequency Mozilla tends to update things, there is always a chance that a new version may break the code. If you run into a situation where it doesn't work on a new or older version of Thunderbird, shoot me an email and I'll see what I can do.<br />
<br />
As always, feedback and suggestions are welcome (If you're nice about it. Otherwise it goes right in the spam folder).<br />
<br />
<a href="https://github.com/mdegrazia/Thunderbird-Email-Parser" target="_blank">Download</a> Thunderbird email parser.<br />
<br />
References:<br />
<br />
1. Library of Congress "<a href="http://www.digitalpreservation.gov/formats/fdd/fdd000385.shtml">Sustainability of Digital Formats Planning for Library of Congress Collections, MOBXRD Email Format</a>."<br />
<br />
2. Mozilla Foundation. "<a href="http://Rebuilding the Global Database">Rebuilding the Global Database"</a><br />
<br /></div>
</div>
Mari DeGraziahttp://www.blogger.com/profile/07035429062451749639noreply@blogger.com3