Reply to post:

This storage startup dedupes what to do what? How?

future research

It will store the files twice, a link to them appropriately. The de-dup table must be able to take both locations of the same hash though as the older flecther4 hashing algorithm used just for checksum would cause problems if de-dup turm on for that pool.

more info can be found at https://blogs.oracle.com/bonwick/entry/zfs_dedup

e.g.

Trust or verify?

If you accept the mathematical claim that a secure hash like SHA256 has only a 2\^-256 probability of producing the same output given two different inputs, then it is reasonable to assume that when two blocks have the same checksum, they are in fact the same block. You can trust the hash. An enormous amount of the world's commerce operates on this assumption, including your daily credit card transactions. However, if this makes you uneasy, that's OK: ZFS provies a 'verify' option that performs a full comparison of every incoming block with any alleged duplicate to ensure that they really are the same, and ZFS resolves the conflict if not. To enable this variant of dedup, just specify 'verify' instead of 'on':

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

SUBSCRIBE TO OUR WEEKLY TECH NEWSLETTER

Biting the hand that feeds IT © 1998–2020