Ross Patterson wrote:
I'm considering merging my disks' shadow files into their
base files. Why?
Exactly my question.
Well, because ... I dunno, it seems like I don't need the
ability to undo literally everything I've ever done on my
VM/CE system.
"literally everything I've ever done" only applies if you only create a shadow file and then never create another one ever again. Which is fine I suppose.
Me? I create a new set of shadow files whenever I want to do *anything* I'm not sure of. Otherwise, I just continue running with the current set.
But one thing I personally *never* do is modify my original base images. My base images are always unmodified. They always remain as the original, virgin, as-originally-delivered/distributed uncustomized base images.
Whenever I install a new distribution of whatever, I create my base images (which is usually to simply unzip the distribution volumes and then immediately convert them to CCKD64), and then immediately mark them as READ-ONLY.
On the first IPL, the first set of shadow files gets automatically created. I then begin to slowly customize my installation to set it up the way I want it. Once I'm happy with the way things are, I then mark THAT set of shadow files as READ-ONLY as well (which of course is done only after the system has been shutdown and Hercules powered off).
On the next IPL, another new set of shadow files gets automatically created (set #2), and from then on, no more shadow files are created. I have the original base images (marked read-only), shadow file set #1 (with my customizations, marked read-only), and the current set of shadow files (still read-write) where all writes go.
In this way I only have to do ONE backup of the original distribution volume and never again, since they will never change. Same with my first shadow file set too (set #1). They too will never change, so I only have to back them up once too, and never again.
The only files I ever need to backup on a daily basis, is shadow file set #2, where all updates occur. The backup of this shadow files set #2 (as well as set #1 too, but that's unimportant at this point) is always quite small. Usually only a few K or a few MB at most. This makes backups quite fast and convenient. I don't need to be constantly backing up my entire system, since the base images have *already* been backed up once, and have never been changed. They've never been modified. And base images are usually many, many GB in size (whereas shadow files are typically very small).
But if you merge your shadow files into your base image, then you need to do another multi-GB backup of everything again! If you never modify your base images, you don't ever need to do that. You back them up once and never again.
And if you ever need to restore your system from a disaster, you only need to restore your two sets of shadow files, because they're all based on the original unmodified set of base images (as well as the previous set of shadow files too of course).
Now maybe you don't mind wasting time and disk space and are quite happy doing things in the most inefficient way possible, but not me. I prefer to think things through and do things in the most disk-space efficient and time efficient manner. But then maybe that's just me. (No slight or disrespect intended by the previous by the way. Just commenting on the way I'm seeing things.)
BUT ... the Hercules doc (at
390.github.io/html/cckddasd.html#utilities) says "Note: It is
not advisable to merge shadow files back into base images. When
merging shadow files, it is only recommended to merge one set
of shadow files back into the previous set of shadow files
Correct. In my scenario, if I wanted, I could mark my first set of shadow files (set #1) read-write, and then merge my set #1 into my set #1, resulting in having only one set of shadow files again. This might be done as you make changes to your system and wish to "save" those changes permanently. (It also prevents the number of shadow file sets from growing too large too, since you can only have a maximum of 8 sets of shadow files.)
But as I said (and as stated in the documentation), we discourage modifying base dasd images for simple logical reasons.
Can certainly CAN (MAY) merge your shadow files back into your base if you want to. But why in the world would you want to do that? Doing so forces you to do a FULL SYSTEM BACKUP all over again, which wastes time and disk space. Your original backup of your base images immediately become obsolete as soon as you modify your current set of base images. It's much faster and easier to just backup your *current* set of shadow files (set #2 only, since set #1, just like your base images, are read-only and thus are never modified, so you only need to back them up *once* and never again).
"When you merge a shadow file back into the base image, you might
see some error/warning message being issued as a result, which
should be considered an unpreventable side effect of such a merge,
and are completely benign. No file damage has actually occurred."
Correct.
Naturally, I get a little concerned when the authors say "Don't use
this tool we built you, or at least not if you don't want your data
destroyed."
Your data will NOT be destroyed. As the documentation states, some error and/or warning messages might be issued, but the merge *will* be successful.
As long as your system doesn't crash before the merge can complete of course. (i.e. power failure or hardware failure, etc.)
Except in the hardware failure or power failure case, data corruption, when it occurs, *always* occurs when the data is being modified (i.e. updated / written to), but never when it's just being read. So why in the world would you want to risk modifying/re-writing known good data if you don't have to? Create you installation default base images, mark then read-only, and back them up. ONCE. And never again. And just use shadow files from then on. MUCH less risky. MUCH faster backups. MUCH less backup disk space consumed.
I mean, it's a no-brainer!
But I have a hard time understanding how to reconcile the first
sentence that says "Hey! Don't do that!", with the last one
that says "It works just fine!".
Do you now? After reading what I've written above, do you now understand why those two statement are not contradictory at all?
So, what say ye? Merge, or not?
Me? I personally would not *ever* merge updates into my original base image. I'd leave all modifications remain in shadow files for the reasons I've already explained.
You do what you want, Ross. It's your system. Just like when you see a doctor who gives you sound medical advice, it's up to you whether you want to heed it or not.
Good Luck
(if there's anything I've written that's unclear or if you have any questions/concerns about any of it, let me know and I'll try to address them. Thanks.)
--
"Fish" (David B. Trout)
Software Development Laboratories
mail: fish@...
