How to export more than 65,536 lines from Access to Excel

Y

yoshimarine

I am using Office 2003 and am getting data dumps from different systems to do
analysis on, but these files can range from 80,000 lines to 900,000 lines.
So we've been doing the dumps into Access as it can handle that amount of
data, but I need it in Excel to do look at the data. I need a code that
will tell Access that every 65,536 lines to open a new worksheet and place
the next 65,536 lines and on and on until the entire table is exported.
 
J

Jeff Boyce

?Any chance you can do the "look at" inside Access?

?Any chance you could use Access to do some initial slice/dice?

You haven't described what you want to use Excel for other than "to look at
the data". I'm guessing there's more to it than this...

Regards

Jeff Boyce
Microsoft Office/Access MVP
 
Y

yoshimarine

There are 212 column/fields that I do need to look at and see all of the
different values in each one. I will have to do some slice/dice, but I still
need to be able to see all of the data. Obviously I will have to do it on
seperate sheets which is combursome, but more managable than trying to look
at all of the data in Access. Applying autofilters helps view the data in
Excel and its just some much easier to scroll than with Access. I can do
some slice and dice in Access, but I still need to be able to export that to
Excel and view it that way. The analysis that we need to do is to find what
values are in each field, discover the patterns and then identify what the
values mean. We are migrating from one system to another at my job and this
analysis is need. Thank You.
 
G

Gina Whipp

yoshimarine,

Just curious because I see you mention that you need to apply
'autoflters'... Can you apply those filters on export? If filtering the
data makes it shorter then 65,536 rows, would that work?

Also, I found this thread which might help or at the very least point you in
the right direction...

http://www.pcreview.co.uk/forums/thread-1667515.php

--
Gina Whipp

"I feel I have been denied critical, need to know, information!" - Tremors
II

http://www.regina-whipp.com/index_files/TipList.htm
 
Y

yoshimarine

I can't filter the data because it will not be whole. I need to have all the
data exported whole - granted it will be sliced due to the limitations of
Excel. I've tried the VB code from the link, but I haven't used VB in Access
yet (I'm decent with Excel VBA) so I can't get it to actual do anything. Can
you help me get that going?
Thank You,
 
R

Roger Carlson

Well, you haven't said exactly what you're trying to do, but I wouldn't
dismiss Access as an analysis tool so quickly. Excel may be easier for you
because you're familiar with it. But with a little bit of investigation,
you might be able to do everything in Access. You admit it will be
cumbersome with separate Excel sheet. Seems to me it's worth looking into.

--
--Roger Carlson
MS Access MVP
Access Database Samples: www.rogersaccesslibrary.com
Want answers to your Access questions in your Email?
Free subscription:
http://peach.ease.lsoft.com/scripts/wa.exe?SUBED1=ACCESS-L
 
Y

yoshimarine

I have explored the use of Access The problem is trying to scroll through
the tables both across and down. Access hangs up periodically if you scroll
too fast. Also to view all of the different values in each of the 212 fields
I would have to build a query with at least one field to sum and group by the
field with the values I want to see. I then have to copy that column to
Excell and paste into it. Example:

field DCCO = $ amount Field PPR = Document Type

I would sum this field and group by this field

that way I have only one record showing for each value. Now I wouldn't have
to do this for all 212 fields, but it's a heck of a lot. Using Excel I could
apply the autofilters and just open the filter to look at the different
values. I could also see the different combinations that way. Much easier
than having to build query after query after query. Believe me I love Access
more than Excel except for the fact that looking at different combinations of
the raw data is faster in Excel. I have to look through the data to find
commonalities and again it is easier to filter and navigate through an excel
spreadsheet than an Access raw data table, at least with my equipment. Of
course if I had 2007 we wouldn't be having this conversation. Employer's a
little slow with software upgrades. I appreciate the help though.
 
Y

yoshimarine

Gina,

I placed it in a module, and really couldn't figure out how to get it to
run. I used the "Macro" function in Access to create a macro which for the
only open for a module is to open which does run it. Then I tried to use an
event procedure but again nothing triggered it. I am completely stumped so
by just getting me to this point I am greatful.

Thank You,
Brett
 
G

Gina Whipp

yoshimarine,

In the Module WIndow click Ctrl+G and a window should open... type:

?ExportChunks("NameOfYourTable","PathYouWantToExportTo","NameYouWantToGiveYourFile")

then click enter on your Keyboard. Then confirm by looking at the file in
the path you exported to.

--
Gina Whipp

"I feel I have been denied critical, need to know, information!" - Tremors
II

http://www.regina-whipp.com/index_files/TipList.htm
 
Y

yoshimarine

Gina,

I am getting a Compile Error saying Expecting Function or variable when I
hit the enter key. Here is the module coding - the only thing that I changed
was the strTarget = FolderPath line - added the file path to where the export
should go to, and this is what is in the Immediate Window:
?ExportChunks("CA_F4311","C:\Documents and Settings\btyamaj\Desktop","Test
Analysis")

Thank You,

Sub ExportChunks(Source As String, _
FolderPath As String, BaseName As String)

Dim dbD As DAO.Database
Dim rsR As DAO.Recordset
Dim fldF As DAO.Field
Dim lngFN As Long
Dim j As Long
Dim lngChunk As Long
Dim lngChunkCount As Long
Dim strTarget As String
Dim strLine As String

Const MAX_CHUNK = 65536
Const STD_CHUNK = 50000
Const DELIM = Chr(9) 'tab-delimited; change

' argument checking and error trapping omitted

'Get ready
Set dbD = CurrentDb()
Set rsR = dbD.OpenRecordset(Source, dbOpenForwardOnly)
rsR.MoveLast 'ensure we get full record count
If rsR.RecordCount = 0 Then
MsgBox "No records to export", vbOKOnly + vbInformation
Exit Sub
ElseIf rsR.RecordCount <= MAX_CHUNK Then
lngChunk = MAX_CHUNK
Else
lngChunk = STD_CHUNK
End If

rsR.MoveFirst
lngChunkCount = 0
Do Until rsR.EOF 'Outer loop: once per file
'Open output file
j = 0
lngChunkCount = lngChunkCount + 1
strTarget = FolderPath & "C:\Docuements and Settings\btyamaj\Desktop\Test
Analysis" & BaseName _
& Format(lngChunkCount, "00") & ".xls"
lngFN = FreeFile()
Open strTarget For Output As #lngFN

Do Until (j = lngChunk) Or rsR.EOF
'inner loop: once per record
'assemble fields into string
strLine = ""
For Each fldF In rsR.Fields
strLine = strLine & CStr(Nz(fldF.Value, "")) _
& DELIM
Next
Print #lngFN, strLine
j = j + 1
Loop 'inner
Close #lngFN
Loop 'outer

rsR.Close
Set rsR = Nothing
Set dbD = Nothing
End Sub
 
G

Gina Whipp

I did change one thing but I too am getting the same error message... The
line I changed is

Set rsR = dbD.OpenRecordset(Source, dbOpenForwardOnly)

to

Set rsR = dbD.OpenRecordset(Source, dbOpenSnapshot)

So now I am looking into a missing reference because nothing else looks
wrong.

--
Gina Whipp

"I feel I have been denied critical, need to know, information!" - Tremors
II

http://www.regina-whipp.com/index_files/TipList.htm
 
G

Gina Whipp

I have tried to figure out what that vague error message means to no avail
BUT I did find another tidbit that sends it to mulitple sheets in Excel:
http://www.wopr.com/index.php?showtopic=644348

I fear I may be asking for help with John Nurick's code because after I get
rid of error message I get 1 record in numerous files OR I put the code back
and get the vague error message, very disturbing. I must be missing
something.

--
Gina Whipp

"I feel I have been denied critical, need to know, information!" - Tremors
II

http://www.regina-whipp.com/index_files/TipList.htm
 
Y

yoshimarine

Gina,

I made the changes that you made and now I'm getting a different error with
regard to some of the declarations. Specifically with one of the Const:
Const DELIM = Char(9) 'tab-delimited; change

I keep getting the error of a compile error: constant expression required.
I can tell you that the data is different, some is text, some are numbers,
some have both numbers and text strings, some are null, and the lengths are
very different. If this line is trying to use the export through a delimitor
I would rather have it delimited by a comma rather than a tab. Again thanks
for your help.

yoshimarine
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top