Simple Hash algorithm to detect duplicate content

[Arne Vajhøj]

My message column is NTEXT(MAX), and it is going to have articles in
it.
I'll look into this approach....

You mean NTEXT *or* NVARCHAR(MAX) ?

Well - neither can be indexed by SQLServer ...

Arne

 

[Arne Vajhøj]

But that's exactly what an indexed unique constraint would do, but in a
more transparent fashion.

I've only ever had to manually store a hash in a database once, and
that was to effectively hash an unknown-until-execution-time number of
Guids when populating a set of sets.

Databases know how to index text columns. I think it's best to let them
do their job.

SQLServer does not.

To quote from BOL:

#Columns that are of the large object (LOB) data types ntext, text,
#varchar(max), nvarchar(max), varbinary(max), xml, or image cannot be
#specified as key columns for an index.

Arne

 

[Jon Skeet [C# MVP]]

SQLServer does not.

To quote from BOL:

#Columns that are of the large object (LOB) data types ntext, text,
#varchar(max), nvarchar(max), varbinary(max), xml, or image cannot be
#specified as key columns for an index.

That's a pity - and it makes life a bit awkward.

The OP could use a hash and then fetch all values which have the same
hash, then performing the comparison.

 

[Christopher Van Kirk]

That's a pity - and it makes life a bit awkward.

The OP could use a hash and then fetch all values which have the same
hash, then performing the comparison.

Indeed. If you "scroll up" you'll see that this is exactly what I
suggested.

 

[Jon Skeet [C# MVP]]

Indeed. If you "scroll up" you'll see that this is exactly what I
suggested.

True - but before we knew that the database couldn't do it all for the
OP in the first place. Where feasible (i.e. not using NTEXT columns
etc) I'd still favour the unique (indexed) constraint in the DB
approach.