Brian_Westerman wrote:
[...]
But, if you apply monthly or quarterly RSU maintenance,
then you will end up with a greater number of shadow files
over time. I'm not sure what the limit on shadow files
is now, but it was 8 when I set up my standard backup routine.
Obviously, if you apply maintenance and each maintenance
set is a separate Shadow file, you will reach the limit fairly
quickly.
Quite true, BUT... That would *only* occur if you never merged any of your shadow file sets. But again, why would you do that? Not merge your shadow file sets once you were confident everything was good?
For example: You have your base images, read-only, and your first set of shadow files (read-write) containing your customizations. Time passes, with shadow file set #1 (your ONLY shadow file set at the moment) accumulating all of your routine day-to-day activity.
Then IBM releases maintenance, which you want to apply to your system. So what do you do? Answer: after shutting your [Hercules] system down normally, you mark your shadow file set #1 READ-ONLY. Then when you bring up Hercules, shadow file set #2 is created. You apply your IBM maintenance. All maintenance gets written to shadow file set #2 (since it's the current read-write set).
You then test everything and after completing your testing, return back to your normal day-to-day routine.
Time passes some more, with shadow file set #2 (the one that has the maintenance applied) continues to accumulate your day-to-day activity.
Then IBM release *another* set of maintenance that you want to apply to your system. So what do you do?
Well, the first thing I would do is merge shadow file set #2 into shadow file set #1, since you've had an entire month or longer to verify it's good and your system is stable. With Hercules shutdown, you mark shadow file set #1 read-WRITE, and then bring Hercules up again. But before IPLing, you issue your "sf-*" command to merge shadow file set #2 into shadow file set #1. Once complete, shadow file set #2 no longer exists (because it's been merged into set #1). You then power off Hercules, and mark shadow file set #1 READ-ONLY again.
Then you bring Hercules up (which automatically creates a BRAND NEW ("empty" so to speak) shadow file set #2 again), IPL you system and proceed to apply the new maintenance just like you did before.
Lather, rinse, repeat.
For as long as desired.
Result: you will never have more than 2 sets of shadow files! Well below the maximum of 8.
So once again, there is NO NEED to *ever* merge your shadow file sets into your original base images. You original base images should remain UNMODIFIED. Your shadow files, over time, will be constantly changing, but who cares? Each time new maintenance is released (or whenever the frick you want to, really!), you either:
a) Save your current shadow file set by marking it read-only so that a new shadow file set is created.
or:
b) You merge your current shadow file set back into your previous shadow file set, and then continue normally from there.
Note that you will *eventually* need to do b) anyway once you've reached the maximum allowable number of shadow file sets (currently 8), but my point is, you can continue to run your Hercules system essentially forever, applying maintenance month after month after month *without* ever exceeding the maximum number of allowable shadow file sets, *AND* without ever having to merge *anything* into your base images (which I will maintain until the day I die should *NEVER* be modified from their original unmodified as-delivered state).
The "safest" (according to IBM) sequence to apply the RSU
maintenance to z/OS, and the one IBM likes, is to get the
hiper stuff on asap and apply the maintenance at the quarterly
RSU releases, which for 2024 will be (RSU2403, RSU2406 RSU2409
and RSU2412). So even if you were to just use that method,
you wouldn't get very far before you were at the limit.
WRONG. It just depends on your goal.
If your goal is to have each application of maintenance in its own separate shadow file set, then yes, you are obviously going to reach the Hercules limit very quickly. But why would you want to manage your shadow files that way? Why not merge your shadow file sets back into the previous set on routine intervals.
OR... do it your way (keep each new set of maintenance in its own set of shadow files), but merge the last set back into the previous set once the limit is reached? But again, why not merge ALL of your shadow file sets into their previous set, until you have only ONE shadow file set? Then once all sets have been merged back into set #1, mark it read-only again to auto-create set #2 and continue normally from there. (Then at some point merge set #2 back into set #1, etc. Lather, rinse, repeat.)
Methinks you guys are not fully grasping what shadow files ARE. They're like incremental backups.
I don't understand your confusion. Using shadow files seems rather simple and straightforward to me. But then maybe that's because I've been using them for so long.
It's not rocket science we're dealing with here.
Some people don't apply maintenance at all, and some simply load
the new levels that are released periodically, which is likely
the case for Fish,
Correct. I don't have access to maintenance. I receive a new release, get it installed and up and running and customized/configured the way I want it (almost exclusively it's just the networking), and then never f**k with it. I only use it to verify the changes we make to Hercules doesn't break anything. If z/OS IPLs fine and I can logon and do some simple things, I'm happy.
Others, like you, are actually USING z/OS in semi-production mode. Doing shit that z/OS system admins and advanced z/OS users/developers do. That's not me.
But... *neither* should have jack all to do with the way you manage your Hercules shadow files! Just as how you use your real Windows or Linux hosts should have anything to do with your routine daily and monthly system backups. To stand there and say "If you're a game player, you need to backup your system this way, but if you're a software developer or a lawyer or a doctor or a musician, etc, you need to backup your system this other way."
BULLSHIT. You backup your system the same way.
Same thing with shadow files. You manage them the same way.
so his method works for him.
And has been for many, MANY years now.
And the way I'm managing my shadow files *SHOULD* work identically for you or anyone else too.
You certainly don't HAVE to do it my way! Personally I don't give a rat's ass HOW the frick you or anyone else does it. I'm just explaining how *I'M* doing and *why* I'm doing it, and recommending and encouraging others to do it the same way too, based on my years of experience and expertise. It's up to each of you to decide for yourselves whether you want to take my advice or not. Whatever you decide is no skin off my nose.
It just doesn't work for me,
Because you're obviously misunderstanding the way I'm saying you should (IMHO) do it. That's probably my fault. I sometimes have trouble explaining things clearly and concisely. (Especially the concisely part! <g>) It's a personal failing of mine.
but then everyone does things differently so what works for me or
for Fish, might not work for you.
A truer truism has rarely been stated. :)
I have found that merging the shadow files down to just one shadow
is quite simple.
Right! So what's the problem then?
Merging into the base file is where you have to really worry and
I have had a couple times (my fault) where it failed. My solution
(which Fish probably won't like) is that when necessary (or yearly
at the longest) I merge to just the one shadow, and then using
DASDCOPY64 copy the Base and the single shadow and make a new base.
You're right: I don't like it. :(
My golden rule is to NEVER modify the base. EVER EVER EVER.
That gives me the new BASE that I want, plus leaves me with the
backup of the old base and shadow file(s).
Which wastes backup disk space and time doing the dasdcopy64, *AND* runs the risk of data corruption should something go wrong during the dasdcopy64 merge/consolidate.
With my way, neither is required and you *always* have a fully backed up CURRENT system with everything. Always.
(As long as you do daily/regular backups of your most current shadow file set of course, which, being shadow files, are usually very small, with backups typically only taking a few seconds up to a minute or three at most.)
To each their own. <shrug>
I also copy everything every day and my backups are not just
the Hercules data, but everything on my computer.
Ditto.
I have five 4TB NVMe drives (20TB total) and a 20TB HDD.
Lucky you. I have only one 4TB backup drive. But then I also have 2 other 4TB drives I used to make monthly clones of my primary backup drive too. So I guess technically that makes 12TB of backup space, with over a year's worth of backup history. I can restore my system to any state it was in on any day of the past year.
[...]
So, for me, space is not really an issue, but recovery is.
Ditto. But why waste the space if you don't need to?
If I could have more than 8 shadow files
Increasing the limit to an infinite number of shadow files has been on my list of things to do for the past almost 20 years now. Sadly, I never seem to find the time to work on it. Other more important things always seem to jump ahead of it. And besides, if you simply manage your shadow files a little differently, the current limit of 8 is actually quite sufficient IMO.
(I rarely go above 4),
Ditto again. :)
then I would/might change things a little, but it would probably
be less efficient to have more than 8 and that's likely why they
set that limit.
Quite right. The more shadow files you have, the slower your reads are, since each read needs to search the previous shadow file (possibly all the way down to the base image) if the track you want isn't in to the current shadow file. That's the way shadow files work. Just like incremental backups.
But given the peace of mind and ease of use, it's completely worth it IMO.
I'm not saying that you should go out and buy a bunch of hardware,
but based on the way I use Hercules, the way I do things works for me.
If it works for you and you're happy with doing it your way, then that's fine. I'm just explaining for the benefit of others the way *I* use shadow files and why I use them the way I do. Once a person grasps (groks) the concept of shadow files, they use them in whatever way works best for them too. As I said, it's no skin off my nose. I'm just offering what I believe to be is good advice backed up with (what I hope is!) sound reasoning and years of experience. Everyone is free to take it or leave it.
I'm constantly moving between RSU levels and sometimes before
certain hipers, and my way lets me move to just about any maintenance
level of any year back for several years. Most people don't
have a need to do that, but I do.
Everybody's situation and needs are different.
In my case, when the NVMe drives went on a super good sale,
(4TB Western Digital Black drives for right around $125 each)
so I bought a bunch, the same with the 20TB Seagate Ironwolf
Pro drives
Ooooo! Those are GOOD ones! I've heard about them. Top of the line.
(which were also cheap at about $150 each) and the NAS boxes
that ended up costing more than the DASD inside of them.
I'm jealous. :)
--
"Fish" (David B. Trout)
Software Development Laboratories
mail: fish@...
