Wednesday, Mar 17th, 2010
Posted By eoghan

File initialization is a normal Windows file system behavior that can create problems for forensic analysts. We have encountered file initialization behaviors in a number of cases and find that it creates significant confusion if the underlying cause is not understood. In several cases, incomplete file initialization was misinterpret as backdating, and in another matter it hampered data salvaging efforts.



File Initialization
File initialization is a process that Microsoft Windows uses when creating a new file system entry. Basically, when a new file is being created, an appropriate amount of unallocated space is reserved for the data that will be stored in the new file. Under certain circumstances, the storage space reserved for the new file may not be used in its entirety, or at all.

When only a portion of the disk space that was reserved for a new file is used to store data associated with that file, this leaves a discrepancy between the logical file size and the actual amount of data stored in the file. As a result, you can have a file that appears to have a logical size larger than the actual amount of data stored for that file. The space between the end of valid data and the end of file is called uninitialized space.

“In NTFS, there are two important concepts of file length: the End of File (EOF) marker and the Valid Data Length (VDL). The EOF indicates the actual length of the file. The VDL identifies the length of valid data on disk. Any reads between VDL and EOF automatically return 0 in order to preserve the C2 object reuse requirement.” (Microsoft fsutil documentation)





Uninitialized space is similar in concept to file slack except that it is contained within the logical file size. Unlike file slack which is no longer associated with a file, data in uninitialized space is in a kind of limbo, trapped at the end of an allocated file but not actually part of that file.

Figure: Diagram of file with a logical size that is larger than its valid data length, leaving uninitialized space

The effect of file initialization behaviors are most easily demonstrated on Windows XP with fsutil as shown here. First, we create a new file that can contain 1024 bytes:


C:\Test>fsutil file createnew cmdLabs-setvaliddata 1024
File C:\Test\cmdLabs-setvaliddata is created


Then we set the valid data length of the new file to 1000 bytes, which leaves 24 bytes unused at the end of the file.

C:\Test>fsutil file setvaliddata cmdLabs-setvaliddata 1000

Valid data length is changed

NTFS captures the difference between logical file size and valid data length in two MFT fields as shown here:

Figure:MFT entry with logical size and valid data length viewed using X-Ways Forensics

Salvaging Data from File System Limbo
The significance of this from a forensic analysis standpoint is that a file with a valid data length smaller than the logical file size can contain data associated with two files: data associated with the new file (VDL bytes), and data from the old file in uninitialized space (logical file size – VDL bytes).

From a forensic analysis perspective, this uninitialized space can be beneficial. While various disk cleaning utilities can be configured to wipe file slack, they generally do not touch data in uninitialized space. As a result, deleted data can remain in uninitialized space indefinitely, even despite data destruction efforts, and can be salvaged by forensic analysts.

However, this arrangement of data can create complications for forensic analysts, particularly when dealing with larger files that have substantial amounts of uninitialized space. For instance, when carving for certain file types, it is common to export unallocated space. However, any data in uninitialized space will not be included in unallocated space. Similarly, when performing keyword searches, a forensic analyst could incorrectly attribute a hit in the uninitialized space with the new file.

In one case, several approaches were employed in an effort to salvage video fragments:

• examined deleted video files still referenced by file system
• performed file carving on unallocated space only
• processed file slack only for fragments of video files

None of these approaches recovered videos from a time period of interest. It was not until we conducted a forensic analysis of uninitialized space that additional video fragment were found.

Misinterpreting Normal File System Behavior as Backdating

Another complication from a forensic analysis standpoint arises when the file creation process is interrupted before the contents of the file is written to disk, because the new file system entry will point to a cluster that still contains data associated with an older file. When this occurs and a date can be associated with the older file, forensic analysts might think that a newer file was overwritten by an older one. This phenomenon can be misinterpreted as evidence of backdating.

As an example, consider a newly created file that has not been initialized and has not had any associated data saved to disk as shown here using fsutil:

C:\Test>fsutil file createnew cmdLabs-creatnew 1024
File C:\Test\cmdLabs-creatnew is created



When a file is initialized but the associated contents was not written to disk, the initialized file system entry may point to a cluster that contains old data as shown below using EnCase. By default, EnCase shows uninitialized space in blue text. The cluster that was allocated to the new file “cmdLabs-createnew” contains older data (folder entries of files from earlier in January).




Figure: EnCase showing folder entries from early January in the cluster allocated to the new initialized file system entry

This situation can be misinterpreted as backdating if the forensic analyst assumes that the clock had to be set back to the old date when the file contents was saved to disk.

(No Comments)

Friday, May 8th, 2009
Posted By admin

Despite the fact that FAT is one of the earliest file systems that forensic practitioners had to deal with, there are still gaps in knowledge that result in misinterpretation and misrepresentation. A prime example of this is the create time in FAT, which is not calculated correctly by some forensic tools. This issue came up in a recently discussion I had with Geoff Fellows (http://www.f3.org.uk/modules/smartclient/client.php?id=3). Although FAT last write timestamps only have a resolution of 2 seconds, whereas the create time has a resolution of 10 milliseconds that some forensic tools fail to take into account. A difference of milliseconds can be important in some cases, and any calculations based on an incorrect representation of creation timestamps will be incorrect.

The confusion arises from the fact that FAT file systems represent create and last write timestamps slightly differently. Last write timestamps are 32 bit little-endian values, interpreted as follows:

24                 16                8                0
+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+
|Y|Y|Y|Y|Y|Y|Y|M| |M|M|M|D|D|D|D|D| |h|h|h|h|h|m|m|m| |m|m|m|s|s|s|s|s|
+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+

\____________/\________/\_________/ \________/\____________/\_________/
year        month      day        hour       minute       second

Take as an example the following FAT folder entry with the last write date highlighted in bold:

$ icat /dev/sdb1 353884 | xxd
0000000: 2e20 2020 2020 2020 2020 2030 004f b079 . 0.O.y
0000010: 763a 763a 0000 b579 763a a502 0000 0000 v:v:…yv:……
0000020: 2e2e 2020 2020 2020 2020 2010 004f b079 .. ..O.y
0000030: 763a 763a 0000 b079 763a 6605 0000 0000 v:v:…yv:f…..
0000040: 4173 0061 006c 0076 0065 000f 009e 6e00 As.a.l.v.e….n.
0000050: 6500 7700 3400 2e00 6700 0000 6900 6600 e.w.4…g…i.f.
0000060: 5341 4c56 454e 7e31 4749 4620 0075 78b9 SALVEN~1GIF .ux.
0000070: 753a 763a 0000 78b9 753a 9212 c1d4 0000 u:v:..x.u:……
0000080: 4269 0066 0000 00ff ffff ff0f 0014 ffff Bi.f…………
0000090: ffff ffff ffff ffff ffff 0000 ffff ffff …………….

Converting to big-endian gives 3a 75 b9 78, which has the following binary representation:

00111010 01110101 10111001 01111000

This translates to a timestamp of 2009.03.21 23:11:48 as follows:

• 7 bits = 0011101 = 29 years since 1980
• 4 bits = 0011 = 3 months
• 5 bits = 10101 = 21 days
• 5 bits = 10111 = 23 hours
• 6 bits = 001011 = 11 minutes
• 5 bits = 11000 = 24 = 48 seconds

Note that 5 bits cannot store all 60 seconds, so last write timestamps must be incremented in 2 second intervals, and is always an even number of seconds.

Although the create time follows this same general calculation, it uses an additional 8 bits to represent one hundredths of a second. In the same directory listing above, the create timestamp is identical to the last modified timestamp except for an additional byte (75 78 b9 75 3a). The additional byte equates to 117 hundredths of a second, which brings the create time to 2009.03.21 23:11:49.17. Thus, the create time can have odd number of seconds, and has a resolution of 10 milliseconds.

Older versions of XWays Forensics had the option to display timestamps to tenths of a second, and newer versions can be configured with greater precision as shown here:

Note: Updated configuration screenshot showing increased precision in XWays 15.1 SR3 provided by Geoff Fellows.

The create timestamps for files copied onto a thumb drive are shown here in XWays Forensics with tenths of seconds displayed:

However, some other forensic tools like TSK only interpret the first four bytes of the create date-time stamp, limiting the resolution to 2 seconds. This is demonstrated in the following listing of the same files as those listed above using XWays (we have reported this bug and it will be fixed in the next release of TSK).

$ fls -l /dev/sdb1 353884 | awk {‘print $3 “\t“ $10 “\t“ $11 “ “ $12′} | sort –key 2
snake_oil.jpg 2009.03.21 23:03:32 (EDT)
orange-Clark_Stanley_Snake_Oil.png 2009.03.21 23:04:32 (EDT)
orange-OilKingLrg.jpg 2009.03.21 23:04:48 (EDT)
orange-Drs_Mixer.jpg 2009.03.21 23:05:24 (EDT)
orange-Hostetters_Bitters.jpg 2009.03.21 23:05:42 (EDT)
orange-wine_of_tar.jpg 2009.03.21 23:05:54 (EDT)
orange-medsh17.jpg 2009.03.21 23:06:04 (EDT)
orange-miraclecure.jpg 2009.03.21 23:06:14 (EDT)
snakeoil.jpg 2009.03.21 23:07:22 (EDT)
snake-oil2.jpg 2009.03.21 23:08:16 (EDT)
banjsalv.jpg 2009.03.21 23:11:30 (EDT)
salvenew4.gif 2009.03.21 23:11:48 (EDT)
yankdcornsalvelg.jpg 2009.03.21 23:12:36 (EDT)
liniment-for-man-and-beast.jpg 2009.03.21 23:12:50 (EDT)
Thumbs.db 2009.03.22 14:25:12 (EDT)

This issue was recently fixed in EnCase version 6.12, and is documented in the release note as “339: EnCase does not read FAT Create Date correctly (BUGID67).” EnCase version 6.12 now displays the create time to the correct second but not the hundredth of a second.

If a forensic examiner is not aware of this error, it can not only introduce inaccuracies into his/her findings, it can also reflect poorly on his/her expertise in this area.

The fact that this bug has existed in EnCase and other widely used forensic tools clearly demonstrates the need to validate important findings using more than one tool. Furthermore, this issue demonstrates that forensic practitioners cannot simply push buttons and unquestioningly rely on the results, but rather must understand the underlying technology sufficiently to double check what their tools are reporting.

Another area relating to FAT file systems where current literature is incorrect is in file allocation strategies but that discussion is for another time…

(No Comments)