PC Review


Reply
Thread Tools Rate Thread

CSV File reading

 
 
Sara
Guest
Posts: n/a
 
      21st Dec 2006
Hello There,

Can anybody help me in reading through large csv files size of about
200mb, with 200 columns.

I have tried with streamreader object parsing line by line but its bit
slow, does anybody have a better solution.

I dont want to landup using 3rd party component.

Regards,
Saravanan K

 
Reply With Quote
 
 
 
 
Steve Barnett
Guest
Posts: n/a
 
      21st Dec 2006
I've used this - very easy to use and very fast:
http://www.codeproject.com/cs/database/CsvReader.asp

It seems to be supported (they fixed a bug for me) and has source if you
want to "mess". Performance wise, they quote:

"To give more down to earth numbers, with a 45 MB CSV file containing 145
fields and 50,000 records, the reader was processing about 30 MB/sec. So all
in all, it took 1.5 seconds! The machine specs were P4 3.0 GHz, 1024 MB."

HTH
Steve

"Sara" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Hello There,
>
> Can anybody help me in reading through large csv files size of about
> 200mb, with 200 columns.
>
> I have tried with streamreader object parsing line by line but its bit
> slow, does anybody have a better solution.
>
> I dont want to landup using 3rd party component.
>
> Regards,
> Saravanan K
>



 
Reply With Quote
 
RobinS
Guest
Posts: n/a
 
      21st Dec 2006


"Sara" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Hello There,
>
> Can anybody help me in reading through large csv files size of about
> 200mb, with 200 columns.
>
> I have tried with streamreader object parsing line by line but its bit
> slow, does anybody have a better solution.
>
> I dont want to landup using 3rd party component.
>
> Regards,
> Saravanan K
>


Assuming each line has a carriage-return/line-feed at the end
(and it must, or readLine wouldn't work), here's an easy way
to read in a file and split the lines by Crlf into an array
in one fell swoop. I don't know what this would do with your
file, but it's free to try it. :-)

Dim crlfs() as String = {ControlChars.CrLf}
Dim lines() as String = _
File.ReadAllText("c:\data.txt").Split(crlfs, StringSplitOptions.None)
Dim numOfLines = lines.Length

Thanks to Francesco Balena; I got this out
of one of his books, probably the VB2005 Core Reference.

Robin S.


 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Reading a text file and reading HardySpicer Microsoft VB .NET 14 21st Jan 2009 06:17 AM
Reading CSV file using OleDB not reading 1st line tshad Microsoft C# .NET 2 8th Feb 2008 11:32 PM
UnauthorizedAccessException when reading XML files (no problem when reading other file-types) blabla120@gmx.net Microsoft ASP .NET 0 15th Sep 2006 03:08 PM
An Automated process of watching a network file folder, reading a file in it and deleting the file using ASP.NET ? Luis Esteban Valencia Muņoz Microsoft ASP .NET 3 4th Jun 2005 11:56 AM
reading the DB vs. reading a text file...performance preference? Darrel Microsoft ASP .NET 3 11th Nov 2004 02:27 PM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 06:11 AM.