Re: "Waterloo Tapes" in H390-VM group's Files area
On Mon, Apr 1, 2024 at 08:55 AM, Dave Wade wrote:
Just found that. There seems to be some issues with some of the early tapes, but generally PETAPE works “a bit”…
I have found various copies of WATERLOO6 and they all check out the same, via MD5sum . The date is the date of the ABSTRACT ABSTRACT file. abele_waterloo.aws is From Bob's distribution. WATERLOO6.aws is from this group's files. waterloo.aws is from the CBT site. [xmas@vm370dev tapes]$ md5sum -b 1982-05-25.*865a170aea956905447f570a60eaa6c6 *1982-05-25.abele_waterloo.aws865a170aea956905447f570a60eaa6c6 *1982-05-25.WATERLOO6.aws865a170aea956905447f570a60eaa6c6 *1982-05-25.waterloo.aws[xmas@vm370dev tapes]$ ?... Mark S.
|
Re: DMSSMN109S in @@CRT0 running DIFF
On Thu, Mar 14, 2024 at 9:45?AM Paul Edwards < mutazilah@...> wrote: On Thu, Mar 14, 2024 at 10:28 AM, Ross Patterson wrote:
> Where did DIFF MODULE Y come from?
> ...
I'm not 100% sure I was involved in getting a DIFF executable
onto a Y drive.
Quite?by accident, I found that the DIFF MODULE dated 2009-06-06 on the VM/370 CE 1.2 Y disk comes from the 591 disk on the GCCCMS user, which is described in GCCCMS MEMO on its 191 disk as "GCC compiler version 3.2.3 MVS 7.0 from 2009".? No source so far, but baby steps are OK too. :-) ? Regardless, assuming it is indeed the version that I ship, the
latest source is in custom.zip at - look for diffutils*.
Yup, I see that.? It'll give me something to play with in my copious spare time :-)
Thanks, Ross
|
Re: "Waterloo Tapes" in H390-VM group's Files area
Harold Grovesteen wrote: Just curiosity
I realize there seems to be some heritage involved in all of this,
but does anyone know which tool creates the garbage? Very old versions of Hercules itself could indirectly create such tapes. The problem (bug) would occur when a previously used tape file (i.e. a non-empty .aws/.het file) was written to. Hercules would write the new data to it, but failed to do a "ftruncate" after each write, thereby leaving whatever old data that happened to still be there at that file displacement immediately follow that newly written tape block: * so if your program was trying to process all physical data on the tape (i.e. if the program was purposely ignoring the 2 or more tapemarks that typically marked the end of data on the tape, it would get an error after reading the last block that was written since the old data (i.e. garbage) that happened to be there wasn't recognized as a valid .aws/.het control block (because it was random previously written tape data). I also seem to recall there was some (also buggy) tool on cbttape.org (I forget which one; some conversion tool I think? Something about 80 byte records?) that would also cause it to happen. But I'm not 100% certain of that since it was so long ago. Is there a tool that removes the unexpected content from the AWS
tape file? My AWSBrowse product comes with one called AWSFixTape: NAME AWS/HET/FKT Tape Fixer Utility, Version: 1.0.0.5961 Copyright (C) 2020-2023, Software Development Laboratories SYNOPSIS AWSFixTape64 [OPTIONS] file ARGUMENTS file The name of the file(s) to be processed. May be specified as a wildcard pattern. OPTIONS /r read-only (don't fix any broken files) /k keepgoing (do not abort if any errors) /s subdirs (process subdirectories too) /v verbose (list *ALL* files processed) /q quiet (don't list files processed) /p progress (/q progress indicator only) WARNING! If you use a wildcard pattern as your filename and/or specify the /s subdirectory option make VERY sure all of the files to be processed are actually AWS, HET or FKT emulated tape files! ALL files that start with a valid 6-byte header will be processed and if the file is not a tape file it will be 'fixed', resulting in a damaged file if the file is not an actual emulated tape! For example, if the file is an operating system control file that by coincidence, begins with a sequence of 6 binary bytes that looks identical to an emulated tape file tapemark block then it will be processed as a tape file and 'fixed'!! (resulting in a damaged operating system file) Be VERY CAREFUL when using wildcards and/or the '/s' subdirectory option! YOU HAVE BEEN WARNED! EXIT STATUS -1 Unrecoverable error (syntax, I/O, etc). n Number of files fixed or needing fixed. Hope that helps! -- "Fish" (David B. Trout) Software Development Laboratories mail: fish@...
|
Re: "Waterloo Tapes" in H390-VM group's Files area
Mark, Just found that. There seems to be some issues with some of the early tapes, but generally PETAPE works “a bit”… Dave ?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]> On Behalf Of Mark Waterbury
Sent: Monday, April 1, 2024 2:40 PM
To: [email protected]
Subject: Re: [h390-vm] "Waterloo Tapes" in H390-VM group's Files area? Hi, Dave,
The old WATERLOO tapes used "PETAPE" (Perkin-Elmer's locally modified version of the original CMS TAPE command).? For those versions of the SHARE VM Project Waterloo tapes, they also included a copy of PETAPE MODULE on the first file on the tape, e.g. along with ABSTRACT ABSTRACT.? So, after doing a TAPE LOAD * * when positioned at file 1, you now have on your A disk a copy of the PETAPE MODULE.? So then, you should just use e.g. PETAPE LOAD etc. for the rest of that particular volume of the WATERLOO SHARE VM project tapes.? Starting with the Waterloo tapes for VM/370 Rel. 6, IIRC, they switched over to using the IBM supplied VMFPLC2 command.
As far as I know, none of the SHARE Project Waterloo tapes ever used BLOCKTAP?
Looking at WATERLOO2.aws, WATERLOO3.aws, and WATERLOO5.aws files, they all end with three consecutive tape marks.? However, I notice that the WATERLOO6.aws file ends with just a single tape mark.
Mark S. Waterbury
|
Re: "Waterloo Tapes" in H390-VM group's Files area
Dave, this leads to a couple more questions...
Is Bloktap part of the R6 / CE distribution?? I can sort of
understand it being there if the source is available so that
someone might fix it.
While I know from the posts that the messed up tapes "leave
garbage at the end of the file",? are the messed up tapes
worth "fixing"?? Is the tape content available in valid formats
elsewhere?
Harold Grovesteen
On 4/1/24 08:24, Dave Wade wrote:
toggle quoted message
Show quoted text
Harold,
After
looking back through the archives, it seems these were
created with Bloktap which doesn’t work on R6.
Dave
?
?
Just curiosity
I realize there seems to be some heritage involved in all of
this, but does anyone know which tool creates the garbage?
Is there a tool that removes the unexpected content from the
AWS tape file?
Harold Grovesteen
On 4/1/24 00:03, Dave Wade wrote:
Mark,
I’ll
check these later, but many AWS tapes end in errors
because one of the tools used to create them leaves
garbage at the end of the file.
Dave
?
?
On Fri, Jan 13, 2023 at 02:39 AM, Dave
Wade wrote:
I imagine from a real tape which came
with the tapes used to build the original system. From
what I remember, and it's a fuzzy memory, the process
was that each site that received a copy agreed to make
two copies for other sites. So most sites had a copy of
the tape.
I was scanning through the WATERLOO
tapes which I got from here. WATERLOO2 and WATERLOO6 end
with errors. Does any one have an alternate source for
these?
?... Mark S.
|
Re: "Waterloo Tapes" in H390-VM group's Files area
Hi, Dave,
The old WATERLOO tapes used "PETAPE" (Perkin-Elmer's locally modified version of the original CMS TAPE command).? For those versions of the SHARE VM Project Waterloo tapes, they also included a copy of PETAPE MODULE on the first file on the tape, e.g. along with ABSTRACT ABSTRACT.? So, after doing a TAPE LOAD * * when positioned at file 1, you now have on your A disk a copy of the PETAPE MODULE.? So then, you should just use e.g. PETAPE LOAD etc. for the rest of that particular volume of the WATERLOO SHARE VM project tapes.? Starting with the Waterloo tapes for VM/370 Rel. 6, IIRC, they switched over to using the IBM supplied VMFPLC2 command.
As far as I know, none of the SHARE Project Waterloo tapes ever used BLOCKTAP?
Looking at WATERLOO2.aws, WATERLOO3.aws, and WATERLOO5.aws files, they all end with three consecutive tape marks.? However, I notice that the WATERLOO6.aws file ends with just a single tape mark.
Mark S. Waterbury
|
Re: "Waterloo Tapes" in H390-VM group's Files area
Sorry, I meant 1.2? VMCE has BLOCKTAP included. Bob Bolch
BLOCKTAP runs on VMCE release 1.3
Bob Bolch
Harold, After looking back through the archives, it seems these were created with Bloktap which doesn’t work on R6. Dave ? ? Just curiosity I realize there seems to be some heritage involved in all of this, but does anyone know which tool creates the garbage? Is there a tool that removes the unexpected content from the AWS tape file? Harold Grovesteen On 4/1/24 00:03, Dave Wade wrote: Mark, I’ll check these later, but many AWS tapes end in errors because one of the tools used to create them leaves garbage at the end of the file. Dave ? ? On Fri, Jan 13, 2023 at 02:39 AM, Dave Wade wrote: I imagine from a real tape which came with the tapes used to build the original system. From what I remember, and it's a fuzzy memory, the process was that each site that received a copy agreed to make two copies for other sites. So most sites had a copy of the tape.
I was scanning through the WATERLOO tapes which I got from here. WATERLOO2 and WATERLOO6 end with errors. Does any one have an alternate source for these?
?... Mark S.
|
Re: "Waterloo Tapes" in H390-VM group's Files area
BLOCKTAP runs on VMCE release 1.3
Bob Bolch
Harold, After looking back through the archives, it seems these were created with Bloktap which doesn’t work on R6. Dave ? ? Just curiosity I realize there seems to be some heritage involved in all of this, but does anyone know which tool creates the garbage? Is there a tool that removes the unexpected content from the AWS tape file? Harold Grovesteen On 4/1/24 00:03, Dave Wade wrote: Mark, I’ll check these later, but many AWS tapes end in errors because one of the tools used to create them leaves garbage at the end of the file. Dave ? ? On Fri, Jan 13, 2023 at 02:39 AM, Dave Wade wrote: I imagine from a real tape which came with the tapes used to build the original system. From what I remember, and it's a fuzzy memory, the process was that each site that received a copy agreed to make two copies for other sites. So most sites had a copy of the tape.
I was scanning through the WATERLOO tapes which I got from here. WATERLOO2 and WATERLOO6 end with errors. Does any one have an alternate source for these?
?... Mark S.
|
Re: "Waterloo Tapes" in H390-VM group's Files area
Harold, After looking back through the archives, it seems these were created with Bloktap which doesn’t work on R6. Dave ?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]> On Behalf Of Harold Grovesteen
Sent: Monday, April 1, 2024 1:15 PM
To: [email protected]
Subject: Re: [h390-vm] "Waterloo Tapes" in H390-VM group's Files area? Just curiosity I realize there seems to be some heritage involved in all of this, but does anyone know which tool creates the garbage? Is there a tool that removes the unexpected content from the AWS tape file? Harold Grovesteen On 4/1/24 00:03, Dave Wade wrote: Mark, I’ll check these later, but many AWS tapes end in errors because one of the tools used to create them leaves garbage at the end of the file. Dave ? ? On Fri, Jan 13, 2023 at 02:39 AM, Dave Wade wrote: I imagine from a real tape which came with the tapes used to build the original system. From what I remember, and it's a fuzzy memory, the process was that each site that received a copy agreed to make two copies for other sites. So most sites had a copy of the tape.
I was scanning through the WATERLOO tapes which I got from here. WATERLOO2 and WATERLOO6 end with errors. Does any one have an alternate source for these?
?... Mark S.
|
Re: "Waterloo Tapes" in H390-VM group's Files area
Just curiosity
I realize there seems to be some heritage involved in all of
this, but does anyone know which tool creates the garbage?
Is there a tool that removes the unexpected content from the AWS
tape file?
Harold Grovesteen
On 4/1/24 00:03, Dave Wade wrote:
toggle quoted message
Show quoted text
Mark,
I’ll
check these later, but many AWS tapes end in errors because
one of the tools used to create them leaves garbage at the
end of the file.
Dave
?
?
On Fri, Jan 13, 2023 at 02:39 AM, Dave
Wade wrote:
I imagine from a real tape which came
with the tapes used to build the original system. From
what I remember, and it's a fuzzy memory, the process was
that each site that received a copy agreed to make two
copies for other sites. So most sites had a copy of the
tape.
I was scanning through the WATERLOO tapes
which I got from here. WATERLOO2 and WATERLOO6 end with
errors. Does any one have an alternate source for these?
?... Mark S.
|
Re: "Waterloo Tapes" in H390-VM group's Files area
Mark, I’ll check these later, but many AWS tapes end in errors because one of the tools used to create them leaves garbage at the end of the file. Dave ?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]> On Behalf Of Mark A. Stevens via groups.io
Sent: Sunday, March 31, 2024 11:48 PM
To: [email protected]
Subject: Re: [h390-vm] "Waterloo Tapes" in H390-VM group's Files area? On Fri, Jan 13, 2023 at 02:39 AM, Dave Wade wrote: I imagine from a real tape which came with the tapes used to build the original system. From what I remember, and it's a fuzzy memory, the process was that each site that received a copy agreed to make two copies for other sites. So most sites had a copy of the tape.
I was scanning through the WATERLOO tapes which I got from here. WATERLOO2 and WATERLOO6 end with errors. Does any one have an alternate source for these?
?... Mark S.
|
Re: "Waterloo Tapes" in H390-VM group's Files area
The other day I was browsing through the messages here and the message concerning Monitor files had this link:
Some of these files are Waterloo tapes, while others are from VM Workshops.
I downloaded those files. However, modern browsers will complain about security, which you will need to override.
Some of those files are already AWS files while others are ZIP files which will need to be unZIPped.
I am embarrassed to state that I have not yet installed Hercules and VM, so I don't know if any of these files will avoid the problems discussed.
When I use my UltraEdit utility to browse some of these files and change to EBCDIC, they do look like what I remember of VMFPLC2 files.
? ? ? Allan
|
Re: "Waterloo Tapes" in H390-VM group's Files area
Hello! There's another problem folks. And it is not the fault of the three here named Dave, it is the problem of the maintainers of the CBT site. That page referenced earlier is now out of date. Oh the original collection of items for VM/370 Release 6 that I still use to confirm that my Hercules setup works might be correct. but the rest isn't I believe. ----- Gregg C Levine gregg.drwho8@... "This signature fought the Time Wars, time and again." On Sun, Mar 31, 2024 at 6:48?PM Mark A. Stevens via groups.io <marXtevens@...> wrote:
On Fri, Jan 13, 2023 at 02:39 AM, Dave Wade wrote:
I imagine from a real tape which came with the tapes used to build the original system. From what I remember, and it's a fuzzy memory, the process was that each site that received a copy agreed to make two copies for other sites. So most sites had a copy of the tape.
I was scanning through the WATERLOO tapes which I got from here. WATERLOO2 and WATERLOO6 end with errors. Does any one have an alternate source for these?
... Mark S.
"The Jedi Knight and the Timelord and their Wookiee guide stood there, the two with their lightsabers drawn and running, The Wookiee with his bowcaster (crossbow) ready, Suddenly the great doors to the building opened and a flurry of Separatists ran out screaming and firing blasters at the two, they caught and trapped all the bolts, the bowcaster fired explosive quarrels. The last two ran out, and fired their blasters at the two, and the bolts were again trapped, first the Timelord gestured and the blaster was grabbed by him, and he stuck it someplace, and then the Jedi Knight did the same thing. The two Separatists ran to them screaming, the Timelord used his mastery of other worldly martial arts to stop him, the Jedi Knight did the same thing, not to be outdone. By the time it was all over they had taken the planet of GuyKin without a loss of life on their side." From a Timelord with the Jedi Knights during the Clone Wars, an as yet unpublished memoir.
|
Re: "Waterloo Tapes" in H390-VM group's Files area
On Fri, Jan 13, 2023 at 02:39 AM, Dave Wade wrote:
I imagine from a real tape which came with the tapes used to build the original system. From what I remember, and it's a fuzzy memory, the process was that each site that received a copy agreed to make two copies for other sites. So most sites had a copy of the tape.
I was scanning through the WATERLOO tapes which I got from here. WATERLOO2 and WATERLOO6 end with errors. Does any one have an alternate source for these? ?... Mark S.
|
Re: BREXX installation instructions
On May 3, 2020, Peter Farley wrote:
>Everything VMARC can be found on Ross's website:
>
Several months later, I put all that stuff up on GitHub at?, because OOCities was a read-only mirror of the GeoCities site I built back around 2002 :-)? I did not put Peter's stuff up there (except for VMARC V1R2P028, his VM/370 fix), or any of the several other VMARC variants, because I wasn't clear on the ownership of them.? As noted in that repo, VMARC is not mine - John Fisher at Rensselaer Polytechnic Institute wrote it and shared it with the world-wide VM community, way back in 1989.? The license is not GPL or any other Open Source, because none of those models existed yet.? I contributed and collected a variety of fixes, but it's really John's work.
Ross
|
The following files and folders have been uploaded to the Files area of the [email protected] group.
By: Harold Grovesteen <h.grovsteen@...>
Description:
Sample of SATK stand-alone dump tool output. Includes the assembly of the bare-metal program that was dumped.
|
Re: Virtual Machine Memory
Thanks!
On 3/27/24 10:16, Dave Wade wrote:
toggle quoted message
Show quoted text
Harold,
I
think I turned attachments off. Just create a temp folder in
the files section and upload, then only those who need it
download it.
Dave
?
?
Thanks so much for your responses.
My first question, can the VM's memory contents be
"extracted" from the virtual machine and this content be moved
from the VM system for processing is in fact possible.? Is it
worth doing?? It is clear if I were to go in that direction,
it would be a separate utility.
As is frequent with such questions, the concept got expanded
somewhat.? Happy to see it stimulated a few "grey/gray cells".
?
I appreciate the suggestion of how to do this with a VM
system.? However, I do not have a running VM system.? I gave
up all personal interest in system administration of all sorts
by the time I retired.? If someone is willing to produce such
a dump in a form my Linux PC could read, I would appreciate
it.? If not I understand that too.? It would give me an idea
of whether this separate utility might be worth looking at
developing.
Just a little background:
I have been working on a bare-metal development system (SATK)
used outside of a mainframe.? My frustrations working on
OpenSolaris support in Hercules drove me to realizing this
tool would be of value.? A lot of Hercules tests now use
SATK.? My work with QEMU finally drove me to work on closing
the loop by producing a memory dump for this development
system.
This is unlike your usual experience where everything is
developed within VM (CMS), run on VM, and the output examined
on VM.? In the SATK environment the bare-metal program is
build outside the mainframe, loaded into it, and the output's
destination, in this case a dump, is also outside the
mainframe, the PC.
Again, thanks so much.? I have a fully working version for
the S/370 world.? Does anyone know if this list supports
attachments?
Harold Grovesteen
On 3/27/24 04:07, Frank D. Engel, Jr.
wrote:
Wondering if the proposed new command could actually be
added at the CP level instead of the CMS/MVS level.
CP might save the state of the machine to a prepared
location in DASD then restore it when the user next logs in,
essentially mimicking the VMWare (for example)
suspend/restore capability.? Users can already disconnect
and leave the machine intact to reconnect to it later (even
leaving it running in the background), so from a user
perspective this would extend that with the option to
reconnect to it even after a shutdown / re-ipl of VM, but
without leaving it running in between.
There would obviously be limits to this when working with
dedicated hardware, but for virtualized hardware as with a
typical CMS machine, I would think it could be done (granted
with a fair amount of effort)?
?
On 3/27/24 00:23, Bertram Moshier
wrote:
Hello Harold,
What you desire doesn't exist in
VM/370 today, but it is possible.? Cray Supercomputers
are real memory systems that use swapping to allow
multiple users to use the same real memory space.? IBM
went with paging for its solution to this problem.?
Seymore Cray rejected this method because it added
around seven cycles?to each operation
referencing?memory.
Let us see if I understand you
correctly.? You want to take a snapshot of a virtual
machine's memory (e.g., 0 to maximum storage size of
the machine).? How you accomplish this depends upon
what you want to do with the image and the memory
methodology used by the users' virtual machine
operating system.
There are two methods for
collecting a copy of the memory for offline
examination.? They are:
1) Copy the state of the virtual
machine to a binary file.? What we used for the Cray
IBM VM Station. When an error occurred, the station
would dump all of its memory, registers, etc., to a
file. We had tools to examine the station and even
run it to the point of failure. (We would use the
internal trace to simulate the interrupts and data
transmissions (to and from the Cray and users'
virtual machines).) This improved the station's
stability?so we could catch coding and design issues
before shipping the product to customers.
2)? You could (as Bob suggested)
use the existing CP DUMP command.? A better solution
would be to allow CP to dump the data more than the
printer.
On the other hand, if you want to
stop and restart an operating system without regard to
space or time and support both paging and swapping,
you will need to add a new command to the operating
system. This would be similar to Windows hibernation
but with more functionality.? As this new command is
at the operating system level (e.g., MVS), it would
support both real and virtual environments.
?
?
For my
Stand Alone Tool Kit project I am working on the
ability to
create a formatted memory dump.? It depends upon
preserving the
machine's memory in a host (Windows, Linux, etc.)
platform's host file
system.? With Hercules this is accomplished using the
Hercules command
'savecore'.
SATK is sometimes used with z/VM virtual machines.?
So, I am asking:
is it possible to transfer the contents of a virtual
machine's physical
(absolute storage) memory out of VM (any version,
VM/370 or z/VM) in any
manner at all?
The assumption is that a bare-metal program has been
executing within
the virtual machine, so CMS is out of the picture.?
This would strictly
be CP commands as I understand things.
Assuming the storage contents exists outside of the
virtual machine,
whatever format that turns out to be, can be
manipulated for use by the
dump program.? Separate problem for later.
Note, I am not opposed to a programmatic approach from
within the
virtual machine embedded within the bare-metal program
if required to
get the memory content out of the virtual machine, BUT
only if that is
the only way.
Asking the experts...
Harold Grovesteen
|
Re: Virtual Machine Memory
Harold, I think I turned attachments off. Just create a temp folder in the files section and upload, then only those who need it download it. Dave ?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]> On Behalf Of Harold Grovesteen
Sent: Wednesday, March 27, 2024 2:58 PM
To: [email protected]
Subject: Re: [h390-vm] Virtual Machine Memory? Thanks so much for your responses. My first question, can the VM's memory contents be "extracted" from the virtual machine and this content be moved from the VM system for processing is in fact possible.? Is it worth doing?? It is clear if I were to go in that direction, it would be a separate utility. As is frequent with such questions, the concept got expanded somewhat.? Happy to see it stimulated a few "grey/gray cells". ? I appreciate the suggestion of how to do this with a VM system.? However, I do not have a running VM system.? I gave up all personal interest in system administration of all sorts by the time I retired.? If someone is willing to produce such a dump in a form my Linux PC could read, I would appreciate it.? If not I understand that too.? It would give me an idea of whether this separate utility might be worth looking at developing. Just a little background: I have been working on a bare-metal development system (SATK) used outside of a mainframe.? My frustrations working on OpenSolaris support in Hercules drove me to realizing this tool would be of value.? A lot of Hercules tests now use SATK.? My work with QEMU finally drove me to work on closing the loop by producing a memory dump for this development system. This is unlike your usual experience where everything is developed within VM (CMS), run on VM, and the output examined on VM.? In the SATK environment the bare-metal program is build outside the mainframe, loaded into it, and the output's destination, in this case a dump, is also outside the mainframe, the PC. Again, thanks so much.? I have a fully working version for the S/370 world.? Does anyone know if this list supports attachments? Harold Grovesteen On 3/27/24 04:07, Frank D. Engel, Jr. wrote: Wondering if the proposed new command could actually be added at the CP level instead of the CMS/MVS level. CP might save the state of the machine to a prepared location in DASD then restore it when the user next logs in, essentially mimicking the VMWare (for example) suspend/restore capability.? Users can already disconnect and leave the machine intact to reconnect to it later (even leaving it running in the background), so from a user perspective this would extend that with the option to reconnect to it even after a shutdown / re-ipl of VM, but without leaving it running in between. There would obviously be limits to this when working with dedicated hardware, but for virtualized hardware as with a typical CMS machine, I would think it could be done (granted with a fair amount of effort)? ? On 3/27/24 00:23, Bertram Moshier wrote: Hello Harold, What you desire doesn't exist in VM/370 today, but it is possible.? Cray Supercomputers are real memory systems that use swapping to allow multiple users to use the same real memory space.? IBM went with paging for its solution to this problem.? Seymore Cray rejected this method because it added around seven cycles?to each operation referencing?memory. Let us see if I understand you correctly.? You want to take a snapshot of a virtual machine's memory (e.g., 0 to maximum storage size of the machine).? How you accomplish this depends upon what you want to do with the image and the memory methodology used by the users' virtual machine operating system. There are two methods for collecting a copy of the memory for offline examination.? They are: 1) Copy the state of the virtual machine to a binary file.? What we used for the Cray IBM VM Station. When an error occurred, the station would dump all of its memory, registers, etc., to a file. We had tools to examine the station and even run it to the point of failure. (We would use the internal trace to simulate the interrupts and data transmissions (to and from the Cray and users' virtual machines).) This improved the station's stability?so we could catch coding and design issues before shipping the product to customers. 2)? You could (as Bob suggested) use the existing CP DUMP command.? A better solution would be to allow CP to dump the data more than the printer.
On the other hand, if you want to stop and restart an operating system without regard to space or time and support both paging and swapping, you will need to add a new command to the operating system. This would be similar to Windows hibernation but with more functionality.? As this new command is at the operating system level (e.g., MVS), it would support both real and virtual environments. ? ? For my Stand Alone Tool Kit project I am working on the ability to
create a formatted memory dump.? It depends upon preserving the
machine's memory in a host (Windows, Linux, etc.) platform's host file
system.? With Hercules this is accomplished using the Hercules command
'savecore'.
SATK is sometimes used with z/VM virtual machines.? So, I am asking:
is it possible to transfer the contents of a virtual machine's physical
(absolute storage) memory out of VM (any version, VM/370 or z/VM) in any
manner at all?
The assumption is that a bare-metal program has been executing within
the virtual machine, so CMS is out of the picture.? This would strictly
be CP commands as I understand things.
Assuming the storage contents exists outside of the virtual machine,
whatever format that turns out to be, can be manipulated for use by the
dump program.? Separate problem for later.
Note, I am not opposed to a programmatic approach from within the
virtual machine embedded within the bare-metal program if required to
get the memory content out of the virtual machine, BUT only if that is
the only way.
Asking the experts...
Harold Grovesteen
|
Re: Virtual Machine Memory
Thanks so much for your responses.
My first question, can the VM's memory contents be "extracted"
from the virtual machine and this content be moved from the VM
system for processing is in fact possible.? Is it worth doing?? It
is clear if I were to go in that direction, it would be a separate
utility.
As is frequent with such questions, the concept got expanded
somewhat.? Happy to see it stimulated a few "grey/gray cells". ?
I appreciate the suggestion of how to do this with a VM system.?
However, I do not have a running VM system.? I gave up all
personal interest in system administration of all sorts by the
time I retired.? If someone is willing to produce such a dump in a
form my Linux PC could read, I would appreciate it.? If not I
understand that too.? It would give me an idea of whether this
separate utility might be worth looking at developing.
Just a little background:
I have been working on a bare-metal development system (SATK)
used outside of a mainframe.? My frustrations working on
OpenSolaris support in Hercules drove me to realizing this tool
would be of value.? A lot of Hercules tests now use SATK.? My work
with QEMU finally drove me to work on closing the loop by
producing a memory dump for this development system.
This is unlike your usual experience where everything is
developed within VM (CMS), run on VM, and the output examined on
VM.? In the SATK environment the bare-metal program is build
outside the mainframe, loaded into it, and the output's
destination, in this case a dump, is also outside the mainframe,
the PC.
Again, thanks so much.? I have a fully working version for the
S/370 world.? Does anyone know if this list supports attachments?
Harold Grovesteen
On 3/27/24 04:07, Frank D. Engel, Jr.
wrote:
toggle quoted message
Show quoted text
Wondering if the proposed new command could actually be added
at the CP level instead of the CMS/MVS level.
CP might save the state of the machine to a prepared location
in DASD then restore it when the user next logs in, essentially
mimicking the VMWare (for example) suspend/restore capability.?
Users can already disconnect and leave the machine intact to
reconnect to it later (even leaving it running in the
background), so from a user perspective this would extend that
with the option to reconnect to it even after a shutdown /
re-ipl of VM, but without leaving it running in between.
There would obviously be limits to this when working with
dedicated hardware, but for virtualized hardware as with a
typical CMS machine, I would think it could be done (granted
with a fair amount of effort)?
On 3/27/24 00:23, Bertram Moshier
wrote:
Hello Harold,
What you desire doesn't exist in VM/370 today, but it is
possible.? Cray Supercomputers are real memory systems that
use swapping to allow multiple users to use the same real
memory space.? IBM went with paging for its solution to this
problem.? Seymore Cray rejected this method because it added
around seven cycles?to each operation referencing?memory.
Let us see if I understand you correctly.? You want to
take a snapshot of a virtual machine's memory (e.g., 0 to
maximum storage size of the machine).? How you accomplish
this depends upon what you want to do with the image and the
memory methodology used by the users' virtual machine
operating system.
There are two methods for collecting a copy of the memory
for offline examination.? They are:
1) Copy the state of the virtual machine to a binary
file.? What we used for the Cray IBM VM Station. When an
error occurred, the station would dump all of its memory,
registers, etc., to a file. We had tools to examine the
station and even run it to the point of failure. (We would
use the internal trace to simulate the interrupts and data
transmissions (to and from the Cray and users' virtual
machines).) This improved the station's stability?so we
could catch coding and design issues before shipping the
product to customers.
2)? You could (as Bob suggested) use the existing CP
DUMP command.? A better solution would be to allow CP to
dump the data more than the printer.
On the other hand, if you want to stop and restart an
operating system without regard to space or time and support
both paging and swapping, you will need to add a new command
to the operating system. This would be similar to Windows
hibernation but with more functionality.? As this new
command is at the operating system level (e.g., MVS), it
would support both real and virtual environments.
Bertram Moshir
WB8ERT
For
my Stand Alone Tool Kit project I am working on the ability
to
create a formatted memory dump.? It depends upon preserving
the
machine's memory in a host (Windows, Linux, etc.) platform's
host file
system.? With Hercules this is accomplished using the
Hercules command
'savecore'.
SATK is sometimes used with z/VM virtual machines.? So, I am
asking:
is it possible to transfer the contents of a virtual
machine's physical
(absolute storage) memory out of VM (any version, VM/370 or
z/VM) in any
manner at all?
The assumption is that a bare-metal program has been
executing within
the virtual machine, so CMS is out of the picture.? This
would strictly
be CP commands as I understand things.
Assuming the storage contents exists outside of the virtual
machine,
whatever format that turns out to be, can be manipulated for
use by the
dump program.? Separate problem for later.
Note, I am not opposed to a programmatic approach from
within the
virtual machine embedded within the bare-metal program if
required to
get the memory content out of the virtual machine, BUT only
if that is
the only way.
Asking the experts...
Harold Grovesteen
|
Re: Virtual Machine Memory
Wondering if the proposed new command could actually be added at
the CP level instead of the CMS/MVS level.
CP might save the state of the machine to a prepared location in
DASD then restore it when the user next logs in, essentially
mimicking the VMWare (for example) suspend/restore capability.?
Users can already disconnect and leave the machine intact to
reconnect to it later (even leaving it running in the background),
so from a user perspective this would extend that with the option
to reconnect to it even after a shutdown / re-ipl of VM, but
without leaving it running in between.
There would obviously be limits to this when working with
dedicated hardware, but for virtualized hardware as with a typical
CMS machine, I would think it could be done (granted with a fair
amount of effort)?
On 3/27/24 00:23, Bertram Moshier
wrote:
toggle quoted message
Show quoted text
Hello Harold,
What you desire doesn't exist in VM/370 today, but it is
possible.? Cray Supercomputers are real memory systems that
use swapping to allow multiple users to use the same real
memory space.? IBM went with paging for its solution to this
problem.? Seymore Cray rejected this method because it added
around seven cycles?to each operation referencing?memory.
Let us see if I understand you correctly.? You want to take
a snapshot of a virtual machine's memory (e.g., 0 to maximum
storage size of the machine).? How you accomplish this depends
upon what you want to do with the image and the memory
methodology used by the users' virtual machine operating
system.
There are two methods for collecting a copy of the memory
for offline examination.? They are:
1) Copy the state of the virtual machine to a binary
file.? What we used for the Cray IBM VM Station. When an
error occurred, the station would dump all of its memory,
registers, etc., to a file. We had tools to examine the
station and even run it to the point of failure. (We would
use the internal trace to simulate the interrupts and data
transmissions (to and from the Cray and users' virtual
machines).) This improved the station's stability?so we
could catch coding and design issues before shipping the
product to customers.
2)? You could (as Bob suggested) use the existing CP DUMP
command.? A better solution would be to allow CP to dump the
data more than the printer.
On the other hand, if you want to stop and restart an
operating system without regard to space or time and support
both paging and swapping, you will need to add a new command
to the operating system. This would be similar to Windows
hibernation but with more functionality.? As this new command
is at the operating system level (e.g., MVS), it would support
both real and virtual environments.
Bertram Moshir
WB8ERT
For
my Stand Alone Tool Kit project I am working on the ability to
create a formatted memory dump.? It depends upon preserving
the
machine's memory in a host (Windows, Linux, etc.) platform's
host file
system.? With Hercules this is accomplished using the Hercules
command
'savecore'.
SATK is sometimes used with z/VM virtual machines.? So, I am
asking:
is it possible to transfer the contents of a virtual machine's
physical
(absolute storage) memory out of VM (any version, VM/370 or
z/VM) in any
manner at all?
The assumption is that a bare-metal program has been executing
within
the virtual machine, so CMS is out of the picture.? This would
strictly
be CP commands as I understand things.
Assuming the storage contents exists outside of the virtual
machine,
whatever format that turns out to be, can be manipulated for
use by the
dump program.? Separate problem for later.
Note, I am not opposed to a programmatic approach from within
the
virtual machine embedded within the bare-metal program if
required to
get the memory content out of the virtual machine, BUT only if
that is
the only way.
Asking the experts...
Harold Grovesteen
|
Mark A. Stevens