Backups - The ugly truth

[Pegasus \(MVP\)]

You don't need to do a full verify if you check the tapes every couple
weeks, that might help your backup time. Some backup applications let
you "test" at the end of a scheduled backup, just to make sure the tape
is readable, it's a good way to save time as long as you test the
backups once a week/month.

Testing backups one a week or month might be insufficient. You
could easily miss a bad tape for many months.

If you use one tape each weekday then the following method will keep
the backup run short, yet test every tape at regular intervals:

- Turn off verify most of the time.
- Turn on verify every 8 calendar days, and get the backup batch file
to send the backup report automatically to your EMail address on
this day only.

This method speeds up the backup run, the tapes will last almost
twice as long (when compared to a daily verification run), you
don't get bored reading daily backup reports every day but you
will detect a bad tape fairly quickly.

 

[Pegasus \(MVP\)]

I'll add my vote to the "full backup" lot. Also, I *don't* verify
tapes but do do periodic tests of backups. When I have verified tapes,
I don't recall a failure. Do you actually get many?

Cheers,

Cliff

I get regular failures. Travan tapes, for example, have a suggested
life expectancy of 50 runs. When used once a week, they should last
about 12 months. I find that they last between 1 and two years.

 

[Enkidu]

Testing backups one a week or month might be insufficient. You
could easily miss a bad tape for many months.

Yes, but if someone wants something back from a week ago, do they
really care in most cases if it is from Monday or Tuesday? If they
want one from a month ago, does it matter if it is the October one or
the November one? How about 6 months ago? Sheesh, they''d probably be
satisfied with a spread of three months.

Don't get me wrong - backups should be AS ACCURATE AS POSSIBLE.
However in most cases the need is not to be anywhere as accurate as
that.

This is getting a bit philosophical....

Cheers,

Cliff

 

[Gerhard]

I wonder to what degree this overwhelming enthusiasm for full backups is
colored by the difficulty of doing incrementals, as it was introduced by
W2K? After all, doing an incremental backup with NT4 was no different than
doing a full backup, only shorter.

We are now wandering from a desire to solve the NTBackup problems of W2K to
a wham bam thank you Mam, just use the /um switch and be done with it
argument.

I am going to give that Guid retrieving script a try. It is entirely
possible that it does not work, and that there really is no way to append
backups under W2K. In that case I will be forced to do full backups,
whether I need them or not. And I will end up singing in the chorus.

Gerhard

I'll add my vote to the "full backup" lot. Also, I *don't* verify
tapes but do do periodic tests of backups. When I have verified tapes,
I don't recall a failure. Do you actually get many?

Cheers,

Cliff

 

[Gerhard]

I agree. And it is a calculated risk that I am taking, albeit mandated by
restrictions that surround me. In other words, people above me have decided
that tapes can only go off-site once a week. Do these people realize that
they'll lose a week's worth of work? I think they do.

There is no way that tapes can go to storage daily, we would actually have
to hire someone to do that. And I am afraid that is not possible under
current conditions where 20% of the staff has been let go. (As it is they
don't even always go to storage on a weekly basis now. If one or more
people are out with a cold or flu, the tapes go nowhere)

So if tapes are just going to sit around in the same location where a fire
could destroy the server, then nothing has been accomplished.

As far as a tape going bad during the weekly cycle, yes that happens, and
then you have to start a new tape with a full manual backup. But it does
not happen every week, maybe once or twice a year. Also, these are DLT
tapes, so no, they are not that cheap.

But I appreciate your input, it is an important discussion that we are
having,
Gerhard

 

[Pegasus \(MVP\)]

See below.

I agree. And it is a calculated risk that I am taking, albeit mandated by
restrictions that surround me. In other words, people above me have decided
that tapes can only go off-site once a week. Do these people realize that
they'll lose a week's worth of work? I think they do.

It is your responsibility to point it out to them, in writing. If you
don't then it's your backside that gets kicked.

There is no way that tapes can go to storage daily, we would actually have
to hire someone to do that. And I am afraid that is not possible under
current conditions where 20% of the staff has been let go. (As it is they
don't even always go to storage on a weekly basis now. If one or more
people are out with a cold or flu, the tapes go nowhere)

Many of my clients ask a staff member to keep the most recent tape
at home. Easy, no extra cost involed, as long as you're aware of the
security implications.

So if tapes are just going to sit around in the same location where a fire
could destroy the server, then nothing has been accomplished.
Correct.

As far as a tape going bad during the weekly cycle, yes that happens, and
then you have to start a new tape with a full manual backup. But it does
not happen every week, maybe once or twice a year. Also, these are DLT
tapes, so no, they are not that cheap.

But I appreciate your input, it is an important discussion that we are
having,
Gerhard

We've come a long way since your initial outburst . . .

 

[Enkidu]

I've used incrementals. They are a PIA, especially when the second or
third tape refuses to work on a restore.... If you have a back spread
over 5 tapes that is multiplying the possible points of failures by 5.
OK, probably MORE than enough of that as you say!

Gerhard, if you do get the script to work, please post back here, so
that others can benefit.

Cheers,

Cliff

 

[zn]

Testing backups one a week or month might be insufficient. You
could easily miss a bad tape for many months.

If you use one tape each weekday then the following method will keep
the backup run short, yet test every tape at regular intervals:

- Turn off verify most of the time.
- Turn on verify every 8 calendar days, and get the backup batch file
to send the backup report automatically to your EMail address on
this day only.

This method speeds up the backup run, the tapes will last almost
twice as long (when compared to a daily verification run), you
don't get bored reading daily backup reports every day but you
will detect a bad tape fairly quickly.

How do you configure the backup software to send e-mailed backup reports?

Thanks.

 

[Leythos]

I wonder to what degree this overwhelming enthusiasm for full backups is
colored by the difficulty of doing incrementals, as it was introduced by
W2K? After all, doing an incremental backup with NT4 was no different than
doing a full backup, only shorter.

The difference is not in the backup, but in the restore. The backup time
doesn't mean much as long as it completes in a desired amount of time so
that the next backup can run.

The restore process is a total pain for incremental backups.

 

[Leythos]

Testing backups one a week or month might be insufficient. You
could easily miss a bad tape for many months.

Not if you read the logs, a failed tape will show in the logs.

 

[Pegasus \(MVP\)]

How do you configure the backup software to send e-mailed backup reports?

Thanks.

1. Place the backup command (e.g. ntbackup.exe) into a batch file.
2. Add a Command Prompt mailer command to send the report
(e.g. mailto.exe, sendmail.exe, blat.exe). They are all freely
downloadable from various sites.

 

[Pegasus \(MVP\)]

Not if you read the logs, a failed tape will show in the logs.

It depends. If "verify" is turned on then a failed tape will
most likely be visible in the log. If "verify" is turned off then
tape flaws are often not reported in the report.

 

[zn]

1. Place the backup command (e.g. ntbackup.exe) into a batch file.
2. Add a Command Prompt mailer command to send the report
(e.g. mailto.exe, sendmail.exe, blat.exe). They are all freely
downloadable from various sites.

Thanks Pegasus! I've got to try that.

 

[Leythos]

It depends. If "verify" is turned on then a failed tape will
most likely be visible in the log. If "verify" is turned off then
tape flaws are often not reported in the report.

Logs get written during the write process, a bad tape would show tape
write errors on any quality tape drive.

 

[Gerhard]

Yes, it would be if you do incrementals for weeks on end. I do a full
backup every week, so there are only 4 incrementals to deal with in the
worst case.
Gerhard

 

[Gerhard]

Yes, I apologize for that. I didn't realize that I was talking to real
people when I wrote that initial message. Very knowledgeable and helpful
people, if I may say so.
I think I was screaming at a corporate giant when I wrote that, and I was
obviously very frustrated.
Again, my apologies to all of you who have been extremely helpful. Peace,
Gerhard

 

[Pegasus \(MVP\)]

Logs get written during the write process, a bad tape would show tape
write errors on any quality tape drive.

Would you be interested in one of my backup reports?
It shows a perfect run during the record phase, and
verify errors during the verify phase. The reason is simple:
The backup program does not check what it writes while
recording (except when it encounters certain milestones,
I assume).

 

[Leythos]

Would you be interested in one of my backup reports?
It shows a perfect run during the record phase, and
verify errors during the verify phase. The reason is simple:
The backup program does not check what it writes while
recording (except when it encounters certain milestones,
I assume).

The backup process will tell you if the tape failed, and while it won't
tell you if the tape failed in the middle, it will tell you if the tape
is unwritable at the start of the backup. So, as it stands, while doing
backups without verify, the backup will most likely work if the tape is
good - which is something checked for at the start, and then doing a
test once a week/month to see if you can restore.

On servers I have yet to see a backup that verified perfectly in years.
Almost every backup shows files that are Object in use or Did not verify
correctly, on some file.

I have not had a tape fail in the middle, it always fails at the start.

 

[Gerhard]

The scripts work, two scheduled append jobs just completed successfully. I
pared down the /um script considerably:

ntbackup backup @C:\winnt\backup.bks /v:yes /hcn /m incremental /p "DLT"
/um

But the important thing is that the append script works. Mark Minasi did a
good thing with that. It made sense to me when I first looked at it: Start
with the known value and find the one that is constantly changing. It's
actually a beautiful piece. And it works.

BEGIN/END CALLOUT A is not recognized by the command processor. When I
googled it most of the references were to XML. But it does not break the
execution either. I eventually commented it out.

You are right, rsm is a trip. It is best to use prepared tapes, but I am
not finished experimenting with that. The fact that I am re-using old sql
tapes probably contributed to some of the problems. I'll report on the
eventual outcome of that next week.

It occurs to me that maybe we should put up a FAQ on NTBackup under W2K,
right now there is a lot of information out there by different people, but
it's scattered. I mean this must be a problem for other people as well?

Anyway, I want to wish everyone who participated in this thread a Happy New
Year, and I want to thank you all for sitting it through with me. Thank
you, you guys are great,

Gerhard