Imagine this:
1) You start an offline backup.
2) Someone notices that the db is not running -- so they execute their usual scripts to restart things.
3) Step #1 is to shutdown down any running databases and stop any left over processes. The *scripted* method is: a) kill -9 any running _pro* processes, b) sleep 5 seconds and then do the same for _mpro* processes, do it again just in case you missed any, c) wait 5 more seconds and kill -9 any _sql processes, d) wait 3 seconds and rm *.lk, and then, finally, ipcrm any shared memory segments that might be lying around.
3a) The shutdown script never even attempts a "proshut". The very first thing it does is to "kill -9" stuff.
4) Now you can go ahead and start your db.
5) Notice that in step #3 no attempt is made to stop probkup?
6) After running for some time the db suddenly announces that the .lk file has disappeared and it shuts itself down.
7) When you try to restart it you discover that the bi file timestamp is out of sync with the database.
8) Interestingly -- the last message prior to the .lk file disappearing is that the backup completed.
9) If you restore that backup it is corrupt.
Something that I did not know: probkup does not notice when its .lk file is deleted. It also does not check when it is finished running to see if the .lk file is the same .lk file that it created.
> Something that I did not know: probkup does not notice when its .lk file is deleted. It also does not check when it is finished running to see if the .lk file is the same .lk file that it created.
The same is true, for example, for dbanalys (while dbanalys is a kind of read-only access).
Single-threaded processes do not monitor for status of .lk files.
Why would they or should they? The .lk file is there to protect the database from unauthorized access. You remove the .lk file and bad things will typically happen.
It would be a good idea if they at least noticed that the .lk file being deleted is not the one that the single-user session created and wrote an appropriate .lg message about that.
But the real problem here is the shutdown script.
Another slightly annoying detail that I had not noticed earlier is that probkup does not write anything obvious in the .lg to indicate that a backup is online or offline. There are some subtle differences in the messages but nothing that jumps right out.
I agree completely that removing the .lk file is asking for trouble.
In a way I think it is helpful to have a nice, simple, and easy to reproduce example of one sort of trouble that can ensue.
Scripts should not delete .lk files but they can easy to do this.
It would be hard to do a bad thing if the content of .lk file (265 bytes) would be a part of master block.
Is this about database backups or a way to send the database to the shredder? :-S
The person who wrote the script probably hated Progress. Nice topic for the upcoming PUG! :-)
The offline backup was unusual. Its role in this worst practices story is that it exposed the flaws of the shutdown script.
By the way, I did wrote a script that deletes .lk file - the script builds the indexes after D&L and in parallel it runs tabanalys for table's areas. There are no reasons for offline tabanalys/dbanalys to create .lk file but it does. And dbanalys ignores the -RO option. That is why I was forced to play with .lk file.
Something used in specific circumstances like that (and which is written and used by an expert) is much less worrisome than something that runs every night.
> On Jun 24, 2019, at 9:46 AM, George Potemkin wrote:
>
> wrote a script that deletes .lk file
your use case, though it makes some sense, is not one that was contemplated.
24 years ago (18 Oct 1995) Tom Bascom wrote:
"The .lk file is a loaded pistol stored on a 3 year old's nightstand. (With an extra large trigger that little fingers can pull more easily.)"
Tom, can I use this phrase to make a tattoo? ;-)
|
Sure :)
I remember that thread.
And, as Mike asks, I'd like to see a photo of that.
3 year olds were not among our target customers
:)
They make excellent testers.
>>1) You start an offline backup.
Having experience with other RDBMSes I would say, the word "offline" -- is asking for trouble in the first place. Everything should be online ( backups, schema changes, index rebuilds, db analyses etc). I understand that kill -9 and deleting .lk look bad. But for me the very first thing "offline backup" is looks as bad as the rest.
>> George wrote:
>>It would be hard to do a bad thing if the content of .lk file (265 bytes) would be a part of master block.
The .lk file was implemented in version 5 or may be earlier?
I believe it would be a good idea to reconsider what was done 25-30 years ago and store the contents of .lk somewhere else, like in master block as George suggests. It will eliminate all horror stories with deleting .lk at once. IMHO, it is worth putting on a road map.
i agree that would be better. but, in the event something goes wrong (i.e. crash, etc) and the database cant be started again, there would have to be a utility to clear the info in the master block.
> there would have to be a utility to clear the info in the master block.
That is the point. Utility will do all necessary checks unlike the 'rm' command run by DBA who may not understand the consequences of removing lk file.
the utility will need a -F switch