Scanning hard drive

B

Bela Istok

I redo the code (your code), using Delegates and get a first run of 4
minutes, and second and later run under a minute. (This is fine if you have
multi processor or multi core machine, with only 1 core you don't win much).
I will try to post the code later when have time to clean it a bit.

Regards,

Bela Istok
Bela Istok said:
The first I see in you code is the use of Doubles, you don't need dobles
you only need long types for the size value.

You code run in 4 min. Going to see if can perform better.

Regards,

Bela Istok
Rotsey said:
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}



}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}



Bela Istok said:
What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and every
file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.


Regards,

Bela Istok

Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey
 
B

Bela Istok

Last run with code:
Using delegated function
Start time:10/22/2007 5:13:19 PM
End time:10/22/2007 5:16:58 PM
Total time:00:03:39.6027394
Size1: 769462539; Size2: 2074779247; Size3:52272046924; File Number:169593;
Directory Number:21854;

Code:
Attached.

Regard,

Bela Istok

Bela Istok said:
I redo the code (your code), using Delegates and get a first run of 4
minutes, and second and later run under a minute. (This is fine if you have
multi processor or multi core machine, with only 1 core you don't win
much). I will try to post the code later when have time to clean it a bit.

Regards,

Bela Istok
Bela Istok said:
The first I see in you code is the use of Doubles, you don't need dobles
you only need long types for the size value.

You code run in 4 min. Going to see if can perform better.

Regards,

Bela Istok
Rotsey said:
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}



}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}



What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and
every file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.


Regards,

Bela Istok

Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey
 
R

Rotsey

ok thanks Bela,

I will run it.

When you ran this code, the file system info is cached
so the second time your un it it is much faster because of this.

Were these stats from the first run????



Bela Istok said:
Last run with code:
Using delegated function
Start time:10/22/2007 5:13:19 PM
End time:10/22/2007 5:16:58 PM
Total time:00:03:39.6027394
Size1: 769462539; Size2: 2074779247; Size3:52272046924; File
Number:169593;
Directory Number:21854;

Code:
Attached.

Regard,

Bela Istok

Bela Istok said:
I redo the code (your code), using Delegates and get a first run of 4
minutes, and second and later run under a minute. (This is fine if you
have
multi processor or multi core machine, with only 1 core you don't win
much). I will try to post the code later when have time to clean it a bit.

Regards,

Bela Istok
Bela Istok said:
The first I see in you code is the use of Doubles, you don't need dobles
you only need long types for the size value.

You code run in 4 min. Going to see if can perform better.

Regards,

Bela Istok
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}



}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}



What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and
every file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.


Regards,

Bela Istok

Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey
 
B

Bela Istok

The stats for the first run was the sended the other times takes < 30 sec.

Regards,

Bela Istok

Rotsey said:
ok thanks Bela,

I will run it.

When you ran this code, the file system info is cached
so the second time your un it it is much faster because of this.

Were these stats from the first run????



Bela Istok said:
Last run with code:
Using delegated function
Start time:10/22/2007 5:13:19 PM
End time:10/22/2007 5:16:58 PM
Total time:00:03:39.6027394
Size1: 769462539; Size2: 2074779247; Size3:52272046924; File
Number:169593;
Directory Number:21854;

Code:
Attached.

Regard,

Bela Istok

Bela Istok said:
I redo the code (your code), using Delegates and get a first run of 4
minutes, and second and later run under a minute. (This is fine if you
have
multi processor or multi core machine, with only 1 core you don't win
much). I will try to post the code later when have time to clean it a
bit.

Regards,

Bela Istok
The first I see in you code is the use of Doubles, you don't need
dobles
you only need long types for the size value.

You code run in 4 min. Going to see if can perform better.

Regards,

Bela Istok
OK. Here is the class the scans the drive.

Can you see any issues here?

Bela, would you be able to post your code that scans the drive.

I am scanning 56GB in about 10 minutes.
public struct PieFileInfo

{

public int RowID;

public PieInfoType Type;

public string Name;

public string FullName;

public double Filesize;

public DateTime LastAccess;

public DateTime Created;

public string Extension;

public DateTime LastWrite;

public int ParentFolderRowID;

public string Drive;

public double FileAge1;

public double FileAge2;

public double FileAge3;

public bool IsRoot;

}

public class ScanDrive

{

private DriveData mList = new DriveData();

public DriveData DataList

{

get { return mList; }

set { mList = value; }

}

private string mDrive;

private int mNewFolderID = 0;

public ScanDrive(string drivefolder)

{

mDrive = drivefolder;

mList.Drive = drivefolder;

}

public bool ScanFolder(string folder, int folderID, ref double psize1,
ref double psize2, ref double psize3)

{

DirectoryInfo objDir = new DirectoryInfo(folder);

int newFolderID;

double size1 = 0;

double size2 = 0;

double size3 = 0;

System.DateTime fileDate;

try

{

newFolderID = WriteRow(objDir, folderID);

if (newFolderID == 0)

{

return false;

}

foreach (DirectoryInfo dir in objDir.GetDirectories())

{

if (!ScanFolder(dir.FullName, newFolderID, ref psize1,ref psize2,ref
psize3))

{

return true;

}

size1 += psize1;

size2 += psize2;

size3 += psize3;

}

foreach (FileInfo fle in objDir.GetFiles())

{

WriteRow(fle, newFolderID);

fileDate = GetAgeDate(fle);

if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))

size1 += fle.Length;

else if (fileDate >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))

size2 += fle.Length;

else

size3 += fle.Length;

}

UpdateFolderRow(newFolderID, size1, size2, size3);

psize1 = size1;

psize2 = size2;

psize3 = size3;

return true;

}

catch (Exception)

{

return true;

}



}

private DateTime GetAgeDate(FileInfo f)

{

if (f.CreationTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age1Days * -1))

return f.CreationTime;

else if (f.LastWriteTime >=
DateTime.Today.AddDays(Settings.Instance.SettingsData.Age2Days * -1))

return f.LastWriteTime;

else

return f.LastAccessTime;

}

private int WriteRow(DirectoryInfo dir, int folderID)

{

PieFileInfo info = new PieFileInfo();

if (mNewFolderID == 0)

info.IsRoot = true;

mNewFolderID++;

info.RowID = mNewFolderID;

info.Type = PieInfoType.Folder;

info.FullName = dir.FullName;

info.Name = dir.Name;

info.LastAccess = dir.LastAccessTime;

info.LastWrite = dir.LastWriteTime;

info.Created = dir.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

mList.DataList.Add(info);

return mNewFolderID;

}

private int WriteRow(FileInfo file, int folderID)

{

PieFileInfo info = new PieFileInfo();

info.Type = PieInfoType.File;

info.Name = file.Name;

info.FullName = file.FullName;

info.Filesize = file.Length;

info.Extension = file.Extension;

info.LastAccess = file.LastAccessTime;

info.LastWrite = file.LastWriteTime;

info.Created = file.CreationTime;

info.ParentFolderRowID = folderID;

info.Drive = mDrive;

if (file.LastAccessTime >= DateTime.Today.AddDays(-30))

info.FileAge1 = file.Length;

else if (file.LastAccessTime >= DateTime.Today.AddDays(-60))

info.FileAge2 = file.Length;

else

info.FileAge3 = file.Length;

mList.DataList.Add(info);

return folderID;

}

private void UpdateFolderRow(int folderID, double size1, double size2,
double size3)

{

int index = mList.DataList.FindIndex(delegate(PieFileInfo fi)

{

if (fi.Type == PieInfoType.Folder && fi.RowID == folderID)

return true;

else

return false;

});

PieFileInfo info = mList.DataList[index];

info.FileAge1 += size1;

info.FileAge2 += size2;

info.FileAge3 += size3;

info.Filesize += size1 + size2 + size3;

mList.DataList[index] = info;

}

}



What do you scan? I do a directory and File scan in my 100GB (64GB of
data) hard drive and took 24 seconds. (Scanned every directory and
every file in the drive for info).

PD: for reference 170,187 files and 21,899 folders.


Regards,

Bela Istok

Hi,

I am writing an app that scans hard drives and logs info
about every fine on the drive.

The first iteration of my code used a class and a generic list
to store the data and rhis took 13min on my 60 GB drive.

I wanted it to be quicker.

So the second time I changed the class to a struct and still
with the generic list got the time down to 4 min.

I am wondering if I can improve on this even more??

Anyone know if it would be possible to improve this
using C# or would it need to be wrritten in assembler or the like?

rotsey
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top