FlushFinalBlock method was called twice on a CryptoStream ERROR MESSAGE

[Caroline]

Hello,
I need to encrypt millions of records (32 bytes each single record), and
would like to reuse the CryptoStream object without recreating it every time
inside the loop. Can anybody point to why I am getting this error on the
second loop iteration:

FlushFinalBlock() method was called twice on a CryptoStream. This method can
only be called once.

Here is the sample code:


**
using System.Security.Cryptography;
using System.IO;

**
private void button2_Click(object sender, System.EventArgs e)
{
int i = 0;
try
{

ICryptoTransform ict;
FileStream fs;

System.Security.Cryptography.RijndaelManaged cryptObj = new
RijndaelManaged();
byte[] KEY_128 = {1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6};
byte[] IV_128 = {1,2,3,4,5,6,7,8,9,0,1,2,3,4,5,6};

MemoryStream ms = null;
CryptoStream cs = null;
StreamWriter sw = null;

fs = new FileStream("..\\..\\isbncrypt.txt", FileMode.Create,
FileAccess.ReadWrite, FileShare.ReadWrite);


ict = cryptObj.CreateEncryptor(KEY_128, IV_128);

string s = "";
byte[] buf = new byte[32];

ms = new MemoryStream();
cs = new CryptoStream(ms, ict, CryptoStreamMode.Write);
sw = new StreamWriter(cs);

while ((i < 3)) // millions of records
{
i++;
s = "12345678901234567890123456789012";

sw.Write(s);
sw.Flush();
cs.FlushFinalBlock(); // ERROR ON SECOND ITERATION: ???
//FlushFinalBlock() method was called twice on a CryptoStream.
This method can only be called once.

ms.Position = 0;
ms.Read(buf, 0, 32);
fs.Write(buf, 0, 32);

}
fs.Write(buf, 0, 32);

ms = null;
fs.Close();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message + "*** " + i);
}

}

 

[Jon Skeet [C# MVP]]

I need to encrypt millions of records (32 bytes each single record), and
would like to reuse the CryptoStream object without recreating it every time
inside the loop. Can anybody point to why I am getting this error on the
second loop iteration:

FlushFinalBlock() method was called twice on a CryptoStream. This method can
only be called once.

Yes - as it says, you can only call it once. It signifies the end of
the data for that stream. You should create a new stream on each
iteration of the loop.

 

[Caroline]

Thank you again Jon. I just wanted to know if it was possible to re-use the
stream (more efficient), but since you are saying no I will believe it
Thank you,
Caroline

 

[William Stacey [MVP]]

In your case, you don't need to create a new cryptostream each write (unless
I missed something in your requirements). We can also get rid of things
like the memorystream and do this a bit more directly like so:

private static void DoEncrypt(string path)
{
int i = 0;
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform encryptor = rm.CreateEncryptor();

using ( FileStream fs = new FileStream(path, FileMode.Create,
FileAccess.ReadWrite, FileShare.ReadWrite) )
using ( CryptoStream cs = new CryptoStream(fs, encryptor,
CryptoStreamMode.Write))
using ( StreamWriter sw = new StreamWriter(cs))
{
string s = null;
while ( (i < 30) ) // millions of records
{
i++;
s = "12345678901234567890123456789012";
sw.WriteLine(s);
}
sw.Flush();
cs.FlushFinalBlock();
}
}

private static void DoDecrypt(string path)
{
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform decryptor = rm.CreateDecryptor();

using ( FileStream fs = File.OpenRead(path) )
using ( CryptoStream cs = new CryptoStream(fs, decryptor,
CryptoStreamMode.Read) )
using ( StreamReader sr = new StreamReader(cs) )
{
while(true)
{
string s = sr.ReadLine();
if ( s == null )
break;
Console.WriteLine(s);
}
}
}

Do this help your speed? If your data is binary, you could also get rid of
the streamReader/writer to save another stream indirection. hth

--
William Stacey [MVP]

Thank you again Jon. I just wanted to know if it was possible to re-use
the stream (more efficient), but since you are saying no I will believe it
Thank you,
Caroline

 

[Jon Skeet [C# MVP]]

In your case, you don't need to create a new cryptostream each write (unless
I missed something in your requirements).

I think you *may* have missed this bit:

<quote>
I need to encrypt millions of records (32 bytes each single record),
</quote>

The last part is the important bit - basically each record being
*separately* encrypted, as far as I can see. Of course, I could easily
have missed how your solution gets round it

 

[William Stacey [MVP]]

In this case, it should not matter if it is 30 or 300 million. It is
encrypting a stream of line terminated lines and reads them back.
If OP want to use absolute bytes, then just change the streamreader/writer
to binaryreader/writer like:

private static void DoEncrypt(string path)
{
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform encryptor = rm.CreateEncryptor();
int i = 0;
using ( FileStream fs = new FileStream(path, FileMode.Create))
using ( CryptoStream cs = new CryptoStream(fs, encryptor,
CryptoStreamMode.Write))
{
RNGCryptoServiceProvider rng = new
RNGCryptoServiceProvider();
byte[] buf = new byte[30];
while ( i < 30 ) // millions of records
{
rng.GetBytes(buf);
cs.Write(buf, 0, buf.Length);
i++;
}
}
}

private static void DoDecrypt(string path)
{
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform decryptor = rm.CreateDecryptor();

using ( FileStream fs = File.OpenRead(path) )
using ( CryptoStream cs = new CryptoStream(fs, decryptor,
CryptoStreamMode.Read) )
using ( BinaryReader br = new BinaryReader(cs) )
{
int i = 0;
while(true)
{
byte[] rec = br.ReadBytes(30);
if ( rec == null || rec.Length == 0 )
break;

Console.WriteLine("Rec {0}: {1}", i++,
BitConverter.ToString(rec));
}
}
}

If you want to be able to Seek to a specific record and decrypt it, then you
need to encrypt blocks as standalone blocks with a new CryptoStream each
record.

 

[Caroline]

So that's what you guys mean by the "using clause". I've never seem that
before. Will try this code. What I meant before is each 32 bytes string has
to be encrypted separately, this is a requirement.

This requirement is because this output file that I am generating on the
desktop with the encrypted strings will be read on a PPC, and that will
avoid the initial loading time to unencrypt the entire file at once (will do
string by string as needed).

Today I am trying to not using the CryptoStream at all, just using
TransformBlock to encrypt but haven't had much success unencrypting on the
PPC. I am using a certain Rijndael algorithm implementation on the Pocket PC
that does not allow setting the Padding property.

Any advice would be appreciated. I really need to figure this out.

In your case, you don't need to create a new cryptostream each write
(unless I missed something in your requirements). We can also get rid of
things like the memorystream and do this a bit more directly like so:

private static void DoEncrypt(string path)
{
int i = 0;
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform encryptor = rm.CreateEncryptor();

using ( FileStream fs = new FileStream(path, FileMode.Create,
FileAccess.ReadWrite, FileShare.ReadWrite) )
using ( CryptoStream cs = new CryptoStream(fs, encryptor,
CryptoStreamMode.Write))
using ( StreamWriter sw = new StreamWriter(cs))
{
string s = null;
while ( (i < 30) ) // millions of records
{
i++;
s = "12345678901234567890123456789012";
sw.WriteLine(s);
}
sw.Flush();
cs.FlushFinalBlock();
}
}

private static void DoDecrypt(string path)
{
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform decryptor = rm.CreateDecryptor();

using ( FileStream fs = File.OpenRead(path) )
using ( CryptoStream cs = new CryptoStream(fs, decryptor,
CryptoStreamMode.Read) )
using ( StreamReader sr = new StreamReader(cs) )
{
while(true)
{
string s = sr.ReadLine();
if ( s == null )
break;
Console.WriteLine(s);
}
}
}

Do this help your speed? If your data is binary, you could also get rid
of the streamReader/writer to save another stream indirection. hth

 

[Caroline]

If you want to be able to Seek to a specific record and decrypt it, then

you need to encrypt blocks as standalone blocks with a new CryptoStream
each record.

Yes. I need to be able to Seek to a specific record and decrypt it.
But creating a new CryptoStream hogs too much memory. It's millions of
records!

If you want to be able to Seek to a specific record and decrypt it, then
you need to encrypt blocks as standalone blocks with a new CryptoStream
each record.

 

[Jon Skeet [C# MVP]]

Yes. I need to be able to Seek to a specific record and decrypt it.
But creating a new CryptoStream hogs too much memory. It's millions of
records!

There should be no reason why creating lots of CryptoStreams should
give you problems - you only need to have one in memory at a time,
after all.

 

[Caroline]

Ok. Call me stupid. I'm not an "MVP". I am a beginner. I missed your post
when you said I need to use this "using" word. All I know is that the memory
on the Task Manager was saying 500 MB usage, I don't know why. I need to
figure out the problem with my code. Maybe I missed on what you said before.
But I will post after I try all these things you've said. Thank you for your
help so far.

 

[William Stacey [MVP]]

Lets back up and slow down. Based on what you have said, the last code I
gave you should work. Please look at it again below. Your *not* decrypting
the entire file at once, your only decrypting the 32 bytes at a time as you
walk down the stream. The br.ReadBytes(32) reads 32 bytes from the
cryptostream. You can then process those bytes as you need and loop around
again until you process all the bytes (ReadBytes returns 0). Simple, fast,
and only requires the one CryptoStream object (one for encrypt and one for
decrypt). It does not matter if your processing 30 records or 300 million,
this will handle any amount of records. I just tested this with 2 million
records in about 8.5 on my Laptop with not noticeable diff in PageFile
usage.

private static void DoEncrypt(string path)
{
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform encryptor = rm.CreateEncryptor();
int i = 0;
using ( FileStream fs = new FileStream(path, FileMode.Create))
using ( CryptoStream cs = new CryptoStream(fs, encryptor,
CryptoStreamMode.Write))
{
RNGCryptoServiceProvider rng = new
RNGCryptoServiceProvider();
byte[] buf = new byte[32];
while ( i < 30 ) // millions of records
{
rng.GetBytes(buf);
cs.Write(buf, 0, buf.Length);
i++;
}
}
}

private static void DoDecrypt(string path)
{
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform decryptor = rm.CreateDecryptor();

using ( FileStream fs = File.OpenRead(path) )
using ( CryptoStream cs = new CryptoStream(fs, decryptor,
CryptoStreamMode.Read) )
using ( BinaryReader br = new BinaryReader(cs) )
{
int i = 0;
while(true)
{
byte[] rec = br.ReadBytes(32);
if ( rec == null || rec.Length == 0 )
break;

Console.WriteLine("Rec {0}: {1}", i++,
BitConverter.ToString(rec));
}
}
}

--
William Stacey [MVP]

So that's what you guys mean by the "using clause". I've never seem that
before. Will try this code. What I meant before is each 32 bytes string
has to be encrypted separately, this is a requirement.

This requirement is because this output file that I am generating on the
desktop with the encrypted strings will be read on a PPC, and that will
avoid the initial loading time to unencrypt the entire file at once (will
do string by string as needed).

Today I am trying to not using the CryptoStream at all, just using
TransformBlock to encrypt but haven't had much success unencrypting on the
PPC. I am using a certain Rijndael algorithm implementation on the Pocket
PC that does not allow setting the Padding property.

Any advice would be appreciated. I really need to figure this out.

In your case, you don't need to create a new cryptostream each write
(unless I missed something in your requirements). We can also get rid of
things like the memorystream and do this a bit more directly like so:

private static void DoEncrypt(string path)
{
int i = 0;
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform encryptor = rm.CreateEncryptor();

using ( FileStream fs = new FileStream(path, FileMode.Create,
FileAccess.ReadWrite, FileShare.ReadWrite) )
using ( CryptoStream cs = new CryptoStream(fs, encryptor,
CryptoStreamMode.Write))
using ( StreamWriter sw = new StreamWriter(cs))
{
string s = null;
while ( (i < 30) ) // millions of records
{
i++;
s = "12345678901234567890123456789012";
sw.WriteLine(s);
}
sw.Flush();
cs.FlushFinalBlock();
}
}

private static void DoDecrypt(string path)
{
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
RijndaelManaged rm = new RijndaelManaged();
rm.Key = key;
rm.IV = iv;
ICryptoTransform decryptor = rm.CreateDecryptor();

using ( FileStream fs = File.OpenRead(path) )
using ( CryptoStream cs = new CryptoStream(fs, decryptor,
CryptoStreamMode.Read) )
using ( StreamReader sr = new StreamReader(cs) )
{
while(true)
{
string s = sr.ReadLine();
if ( s == null )
break;
Console.WriteLine(s);
}
}
}

Do this help your speed? If your data is binary, you could also get rid
of the streamReader/writer to save another stream indirection. hth

--
William Stacey [MVP]

Thank you again Jon. I just wanted to know if it was possible to re-use
the stream (more efficient), but since you are saying no I will believe
it
Thank you,
Caroline

I need to encrypt millions of records (32 bytes each single record),
and
would like to reuse the CryptoStream object without recreating it
every time
inside the loop. Can anybody point to why I am getting this error on
the
second loop iteration:

FlushFinalBlock() method was called twice on a CryptoStream. This
method can
only be called once.

Yes - as it says, you can only call it once. It signifies the end of
the data for that stream. You should create a new stream on each
iteration of the loop.

 

[Jon Skeet [C# MVP]]

Ok. Call me stupid. I'm not an "MVP".

No-one's calling you stupid, honestly.

I am a beginner. I missed your post
when you said I need to use this "using" word. All I know is that the memory
on the Task Manager was saying 500 MB usage, I don't know why. I need to
figure out the problem with my code. Maybe I missed on what you said before.
But I will post after I try all these things you've said. Thank you for your
help so far.

It's possible that with your initial code, things were hanging around
too long but that using "using" blocks and avoiding reallocating
buffers could alleviate the problem.

 

[Jon Skeet [C# MVP]]

Lets back up and slow down. Based on what you have said, the last code I
gave you should work. Please look at it again below. Your *not* decrypting
the entire file at once, your only decrypting the 32 bytes at a time as you
walk down the stream. The br.ReadBytes(32) reads 32 bytes from the
cryptostream. You can then process those bytes as you need and loop around
again until you process all the bytes (ReadBytes returns 0). Simple, fast,
and only requires the one CryptoStream object (one for encrypt and one for
decrypt). It does not matter if your processing 30 records or 300 million,
this will handle any amount of records. I just tested this with 2 million
records in about 8.5 on my Laptop with not noticeable diff in PageFile
usage.

I think you're missing the aim - if I need to get to the last record, I
have to decrypt everything up until that point with your scheme. By
using a different CryptoStream each time, I can seek to the right point
and *then* decrypt.

 

[William Stacey [MVP]]

I think you're missing the aim - if I need to get to the last record, I
have to decrypt everything up until that point with your scheme.

Most likely I am missing it. I am not getting getting the full story from
Caroline so I try to read between the lines here. On one hand, she says see
needs to seek, but then talks like she will enumerate all records anyway.
Caroline, how will you access the records 1-1,000,000 or some random order?

--wjs

 

[Jon Skeet [C# MVP]]

Most likely I am missing it. I am not getting getting the full story from
Caroline so I try to read between the lines here. On one hand, she says see
needs to seek, but then talks like she will enumerate all records anyway.
Caroline, how will you access the records 1-1,000,000 or some random order?

I haven't seen where it looks like she's going to be enumerating all
the records, other than in the sample code - which I just took to be
sample code which demonstrated the problem.

Note that this is going to be read on the compact framework by the way,
so even if reading 2 million records only took 8.5 seconds on your
laptop, reading that many on a mobile device could take a *lot* longer.

 

[William Stacey [MVP]]

Note that this is going to be read on the compact framework by the way,

so even if reading 2 million records only took 8.5 seconds on your
laptop, reading that many on a mobile device could take a *lot* longer.

I agree. The seconds was just to give some context. If random is required,
maybe something like this (needs work, but has general idea). Also need to
make sure we use same padding on both systems:

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Security.Cryptography;
using System.Diagnostics;

namespace FileMover
{
public class MyRecords
{
string path;
readonly object syncRoot = new object();
FileStream fs;
RijndaelManaged rm;
ICryptoTransform encryptor;
ICryptoTransform decryptor;
const int recSize = 48;

public MyRecords(string path, byte[] key, byte[] iv)
{
this.path = path;
this.fs = File.Open(path, FileMode.OpenOrCreate,
FileAccess.ReadWrite);
this.rm = new RijndaelManaged();
this.rm.Key = key;
this.rm.IV = iv;
this.encryptor = rm.CreateEncryptor();
this.decryptor = rm.CreateDecryptor();
}

protected void Dispose(bool disposing)
{
try
{
if ( !disposing )
{
return;
}
if ( fs != null )
{
fs.Close();
}
}
finally
{
}
}

public byte[] FindIndex(int index)
{
if ( index < 0 )
throw new ArgumentOutOfRangeException("index");

int offset = index * recSize;
if ( offset > (fs.Length - recSize) )
throw new IOException("index passed end of file.");

fs.Position = offset;
byte[] block = GetBytes(fs, recSize);
byte[] rec = DecryptBlock(this.decryptor, block);
return rec;
}

public void Add(byte[] rec)
{
if ( rec == null || rec.Length != 32 )
throw new ArgumentOutOfRangeException("rec is not valid
size.");
byte[] block = EncryptBlock(this.encryptor, rec, 0, rec.Length);
fs.Write(block, 0, block.Length);
//Console.WriteLine("Block size is:"+block.Length);
}

private static byte[] EncryptBlock(ICryptoTransform encryptor,
byte[] buffer, int offset, int count)
{
if ( buffer == null )
throw new ArgumentNullException("buffer");
if ( (offset < 0) || (offset > buffer.Length) )
throw new ArgumentOutOfRangeException("offset");
if ( (count < 0) || (count > (buffer.Length - offset)) )
throw new ArgumentOutOfRangeException("count");

using ( MemoryStream msEncrypt = new MemoryStream() )
using ( CryptoStream csEncrypt = new CryptoStream(msEncrypt,
encryptor, CryptoStreamMode.Write) )
{
csEncrypt.Write(buffer, offset, count);
csEncrypt.FlushFinalBlock();
byte[] encrypted = msEncrypt.ToArray();
return encrypted;
}
}

private static byte[] DecryptBlock(ICryptoTransform decryptor,
byte[] encrypted)
{
if ( decryptor == null )
throw new ArgumentNullException("decryptor");
if ( encrypted == null )
throw new ArgumentNullException("encrypted");

using ( MemoryStream msDecrypt = new MemoryStream(encrypted) )
using ( CryptoStream csDecrypt = new CryptoStream(msDecrypt,
decryptor, CryptoStreamMode.Read) )
{
byte[] buf = new byte[encrypted.Length];
int read = csDecrypt.Read(buf, 0, buf.Length);
if ( read < buf.Length )
{
byte[] tmpBuf = new byte[read];
Buffer.BlockCopy(buf, 0, tmpBuf, 0, read);
return tmpBuf;
}
return buf;
}
}

private static byte[] GetBytes(Stream stream, int count)
{
int bytesRead = 0;
byte[] buf = new byte[count];
int read = 0;

while ( bytesRead < count )
{
read = stream.Read(buf, bytesRead, count - bytesRead);
if ( read == 0 )
return null; // Socket closed before all bytes read.
bytesRead += read;
}
return buf;
}
}
}

 

[Caroline]

Jon is getting the point.

I will access each encrypted record via a binary search on the pocket pc.
I have tried this and have no problems with it, it's quite fast.

But the problem I am having is creating the encrypted strings using
CryptoStream on my laptop (or desktop). Last night I tried using the close
clause, it's faster, but still took about 18 hours to run. It goes fast in
the beginning, but than after the initial 8 mb it runs slower and slower.
The generated file is 183 MB total. I am going to try using a memory
profiling tool.

 

[William Stacey [MVP]]

Using the class I posted in last post, I created 1 million in 7.3 secs on my
laptop and could find and decrypt record 500,000 in 1.5 milliseconds.

string path = @"\isbncrypt.txt";
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
File.Delete(path);
MyRecords mr = new MyRecords(path, key, iv);
RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
byte[] rec = new byte[32];
Stopwatch sw = new Stopwatch();
sw.Start();
for ( int i = 0; i < 1000000; i++ )
{
rec[0] = (byte)i;
mr.Add(rec);
}
sw.Stop();
Console.WriteLine("Time to create records:" +
sw.Elapsed.TotalSeconds.ToString());

sw.Start();
rec = mr.FindIndex(50000);
sw.Stop();
Console.WriteLine("Rec {0}:{1}", 500000,
BitConverter.ToString(rec));
Console.WriteLine("Time to find:" +
sw.Elapsed.TotalSeconds.ToString());

Time to create records:7.3629251
Rec
500000:20-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-
ms to find:1.5317
ticks to find:15317


--
William Stacey [MVP]

Jon is getting the point.

I will access each encrypted record via a binary search on the pocket pc.
I have tried this and have no problems with it, it's quite fast.

But the problem I am having is creating the encrypted strings using
CryptoStream on my laptop (or desktop). Last night I tried using the close
clause, it's faster, but still took about 18 hours to run. It goes fast
in the beginning, but than after the initial 8 mb it runs slower and
slower. The generated file is 183 MB total. I am going to try using a
memory profiling tool.

 

[Jon Skeet [C# MVP]]

Jon is getting the point.

I will access each encrypted record via a binary search on the pocket pc.
I have tried this and have no problems with it, it's quite fast.

But the problem I am having is creating the encrypted strings using
CryptoStream on my laptop (or desktop). Last night I tried using the close
clause, it's faster, but still took about 18 hours to run. It goes fast in
the beginning, but than after the initial 8 mb it runs slower and slower.
The generated file is 183 MB total. I am going to try using a memory
profiling tool.

Can you post a complete program which generates dummy data so that we
could try to improve it? I'd be more than happy to try to tweak it.

 

[Caroline]

Thanks. Will try your code.

Using the class I posted in last post, I created 1 million in 7.3 secs on
my laptop and could find and decrypt record 500,000 in 1.5 milliseconds.

string path = @"\isbncrypt.txt";
byte[] key = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
byte[] iv = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
6 };
File.Delete(path);
MyRecords mr = new MyRecords(path, key, iv);
RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider();
byte[] rec = new byte[32];
Stopwatch sw = new Stopwatch();
sw.Start();
for ( int i = 0; i < 1000000; i++ )
{
rec[0] = (byte)i;
mr.Add(rec);
}
sw.Stop();
Console.WriteLine("Time to create records:" +
sw.Elapsed.TotalSeconds.ToString());

sw.Start();
rec = mr.FindIndex(50000);
sw.Stop();
Console.WriteLine("Rec {0}:{1}", 500000,
BitConverter.ToString(rec));
Console.WriteLine("Time to find:" +
sw.Elapsed.TotalSeconds.ToString());

Time to create records:7.3629251
Rec
500000:20-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-
ms to find:1.5317
ticks to find:15317