Fuzzy Hashing
Hash functions are well-known methods in computer science to map arbitrary large input to bit strings of a fixed length that serve as unique input identifier/fingerprints. A key property of cryptographic hash functions is that even if only one bit of the input is changed the output behaves pseudo randomly and therefore similar files cannot be identified. However, in the area of computer forensics it is also necessary to find similar files (e.g. different versions of a file), wherefore we need a similarity preserving hash function also called fuzzy hash function. In this paper we present a new approach for fuzzy hashing called bbHash. It is based on the idea to ‘rebuild’ an input as good as possible using a fixed set of randomly chosen byte sequences called building blocks of byte length l (e.g. l= 128 ). The proceeding is as follows: slide through the input byte-by-byte, read out the current input byte sequence of length l , and compute the Hamming distances of all building blocks against the current input byte sequence. Each building block with Hamming distance smaller than a certain threshold contributes the file’s bbHash. We discuss (dis- )advantages of our bbHash to further fuzzy hash approaches. A key property of bbHash is that it is the first fuzzy hashing approach based on a comparison to external data structures. Keywords: Fuzzy hashing, similarity preserving hash function, similarity digests, Hamming distance, computer forensics.
Due to the growth, distribution, and usage of electronic device increase over the recent years; it’s no wonder the crime availability has also increased. What I mean here is that with so much electronic usage it is easy for a nasty person to get into trouble (that was technical). I found a paper which had been written back in the day, 2012, but it inspired me to do this article on bbHashing which is related to Fuzzy hashing…..only different.
The principle of this process is built on what is called building blocks. The suspected package is broken into blocks using a mathematical formula, and is run through the process to find matching packets.
Software Deployment with Group Policy
A MSI (installation file) is needed to complete this operation
Configure a share file to store the software package on your C: drive
New folder and call it App Deploy for recognition
Share this folder, give it princaple of least privilege for security (don’t change the permissions
Make note of the pathway to the new file in the properties file for future reference
Add everyone to the read list…this can changed later
Create a new folder for the software to be installed in your App Deploy file, and copy the file into the folder
Create a new Group Policy Organization for the diployment
Ensure the GPO is in the correct location. Putting it in the wrong location can open opportunities for unauthorized use
Go into Group policy management
Select the Organizational Unit which is to be used
New organization
Name it {workstations}
Right click on the object, create the new policy object and name it Software deployment
Click OK
In this section we give a detailed description of our fuzzy hash function bbHash. It works as follows: To find the optimal representation of a given file by the set of building blocks, we slide through the input file byte-by-byte, read out the current input byte sequence of length l , and compute the Hamming distances of all building blocks against the current input byte sequence. If the building block with the smallest Hamming distance is smaller than a certain threshold, too, its index contributes to the file’s bbHash. We write L f for the length of the input file in bytes. The pseudocode of our algorithm bbHash is given in Algorithm 1. It proceeds as follows for each offset i within the input file, 0≤ i≤ Lf − 1− l : If BSi denotes the byte sequence of length l starting at the i -th byte of the input, then the algorithm computes the N Hamming distances of BSi to all N building blocks: hd k ,i = HD(bbk , BSi ) is the Hamming distance of the two parameters bbk and BSi , 0≤ k< N . As the Hamming distance is the number of different bits, we have 0≤ hd k ,i≤ 8⋅ l . In Sec. 3.1 we defined the default length of a building block in bytes to be 128, i.e. we assume l= 128 . As an example HD(bb2, BS100) returns the Hamming distance of the building block bb2 and the bytes B100 to B227 of the input. In other words the algorithm slides through the input, byte-by-byte, and computes the Hamming distance at each offset for all N building blocks like it is given in Fig. 3. The bbHash value is formed by the ordered indicies of triggered building blocks. In order to trigger a building block to contribute to the bbHash, it has to fulfill two further conditions: 1. For a given i (fixed offset), we only make use of the closest building block, i.e. we are looking for the index k with the smallest Hamming distance hd k ,i . Fig. 2: Building block with index 0 Fig. 1. Generation of the building blocks ADFSL Conference on Digital Forensics, Security and Law, 2012 94 2. This smallest hd k ,i also needs to be smaller than a certain threshold t . Each BSi that fulfills both conditions will be called a trigger sequence. To create the final bbHash hash value, we concatenate all indicies k of all triggered building blocks (in case we have two triggered building blocks for BSi , only the smallest index k is chosen)
Software Deployment with Group Policy
A MSI (installatiion file) is needed to complete this operation
Configure a share file to store the software package on your C: drive
New folder and call it App Deploy for recognition
Share this folder, give it principle of least privilege for security (don’t change the permissions)
Make note of the pathway to the new file in the properties file for future reference
Add everyone to the read list…this can be changed later
Create a new folder for the software to be installed in your App Deploy file, and copy the file into the folder
Create a new Group Policy Organization for the deployment
Ensure the GPO is in the correct location. Putting it in the wrong location can open opportunities for unauthorized use
Go into Group policy management
Select the Organizational Unit which is to be used
New organization
Name it {workstations}
Right click on the object, create the new policy object and name it Software Deployment
Click OK
How’s that for a formula? When I spent my first semester here at BTC, part of it was in a class called Computer Forensics. We worked with Wireshark, packet sniffing, password cracking, MD5 and SHA1 hashes, really cool stuff. One of our projects was the Rhino Hunt. We were given a scenario where it was criminal to have more than seven rhino pictures in your possession. Sounds funky, but it was fun.
Software Deployment with Group Policy
A MSI (installation file) is needed to complete this operation
Configure a share file to store the software package on your C: drive
New folder and call it App Deploy for recognition
Share this folder; give it principle of least privilege for security (don’t change the permissions)
Make note of the pathway to the new file in the properties file for future reference
Add everyone to the read list…this can be changed later
Create a new folder for the software to be installed in your App Deploy file, and copy the file into the folder
Create a new Group Policy Organization for the deployment
Ensure the GPO is in the correct location. Putting it in the wrong location can open opportunities for unauthorized use
Go into Group policy management
Select the Organizational Unit which is to be used
New organization
Name it [workstations]
Right click on the object, create the new policy object and name it Software deployment
Click OK
Now in case you are wondering why I have three copies of the same policy. Well, you guessed it. This is an example of a document which is suspected of being stolen (appropriated) from the boss’s office. It may have been altered to hide the theft, but if this document were run through the bbHash process, it would come up with three different The hash value length depends on three different properties: the file size L f , the threshold t and the building block length l . If we expect that both other parameters are fixed, then ñ a larger L f will increase the hash value length as the input is supposed to have more trigger sequences. ñ a higher t will increase the hash value length as more BSi will have a Hamming distance lower than the threshold t . ñ a large l will decrease the performance and the hash value length3 . In order to have full coverage our aim is to have a final hash value where every input
This whole article is full of little surprises isn’t it.
Hackers use this type of deception when infiltrating a system.
If one is interested in this complete article, it can be found at:
Proceedings of the Conference on Digital Forensics, Security and Law. (n.d.). Retrieved May 05, 2016, from http://proceedings.adfsl.org/index.php/CDFSL/article/view/63/61
The following is the title of the paper which was written on bbHashing:
ADFSL Conference on Digital Forensics, Security and Law, 2012 89
A FUZZY HASHING APPROACH BASED ON RANDOM SEQUENCES AND HAMMING DISTANCE
Frank Breitinger & Harald Baier
Center for Advanced Security Research Darmstadt (CASED)
and Department of Computer Science, Hochschule Darmstadt,
Mornewegstr. 32, D – 64293 Darmstadt, Germany,
Mail: {frank.breitinger, harald.baier}@cased.de
This blog is in no way intended to be a copy and paste process. I have a novice knowledge of MD5 and SHA1 hashes and the concept of comparing files for malicious intent. It is just that I found this of interest, and couldn’t in any way, shape, or form put this into words. By the way, have you found the differences in the files? They are very slight, but do exist. Like M&Ms, Santa… Easter…
Hope you all enjoyed, and “Happy Researching”
Comments