The situation with 16M of memory and GCC compilations is a known
consideration. I am not current with this topic although I was very
much aware of what transpired due to the 16M constraint when it was
discovered. If anyone has more current information that would alter
anything I say below please chime in.
Ignore if this entire topic is not of interest to you. My response is
a bit long, and I apologize, but this is not a small topic.
Harold Grovesteen
On Sat, 2022-10-29 at 17:15 +0200, Bernd Oppolzer via groups.io wrote:
I don't have a solution to this problem, obviously, but I would like
to
ask some questions to understand the problem better.
With respect to this: "where it first compiled cRexx on VM/370, now
it
does not" ...
does the storage problem depend on the size or complexity of the
source
code being compiled?
That is: does gcc on VM refuse to start, or does the error occur
later
during compilation?
If, for example, gcc compiles simple programs and does not compile
complex programs,
the error could come from the non-availability of dynamic (free)
storage
used to store
compiler lists, used to store variable names, attributes, offsets
etc.
The issues related to compilation with GCC surrounds the size of the
module being compiled. This was first discovered when trying to
compile GCC with GCC. Prior to this point GCC (actually the i370
target of GCC) was compiled on a PC but output mainframe assembler.
The assembler modules were moved to a Hercules running operating
system where it was assembled and linked. Whether only one GCC module
triggered this situation I do not know. After all, once the first
module was encountered, it was pretty much of a show stopper for GCC
compiling GCC on Hercules.
I would look at what has changed since compilation of cRexx on VM/370
worked vs. now that it does not. Did cRexx change? Did the VM/370
environment change (reducing available memory in the virtual machine)?
Rather than looking for a GCC resolution, looking at cRexx might be the
better approach. Refactoring any cRexx module which can not be
compiled into two smaller modules would probably work. I know nothing
about cRexx source code, so how difficult that might be is unclear to
me. Maybe cRexx is one huge module. This is do not know.
As far as I know, getting past the compilation process should be
sufficient. Although, obviously, at some point source code module
structure changes will not work.
One possible solution is to return to use of i370 on a PC. cRexx could
be compiled on the PC and the assembly modules assembled and linked on
VM/370. Assuming it still exists, how to acquire the version of gcc
i370 that actually runs on a PC I have no idea. Whether there has been
any code generation changes that would make the output incompatible
with what gcc on VM/370 generates, I have no idea either. Or changes
with the C-library used by GCC, no clue. But that is a possible path.
All of the techniques listed below for GCC would also work for cRexx
and that is where I would suggest efforts be applied, not GCC.
The "fix GCC" or "compile on a PC" paths introduce a lot more unknowns
and variables than working with cRexx. Obviously, there might be
reasons for not touching cRexx, but something has to be touched if it
is to be compiled.
My bet, if cRexx has not been touched, either GCC in the version of VM
being used is different or the VM itself is different and that is where
the change occurred. The original poster, Rene, would know what might
have changed in his environment. If there was any, I did not see why
cRexx was being compiled.
If this was the case, then - of course - a reduction in module size
by
using overlay techniques would help ...
or by issuing dynamic loads (and deletes) of compiler components
which
are not used together
at the same time (but at different times during compilation, maybe
by
separating the preprocessor
from the rest of the compilation or by separating the compiler
passes
from each other ...
if there are multiple passes).
AFAIK, the gcc compiler for VM is a simple port of gcc from other
platforms, but nothing has been
done to cope with the special requirements of the 16 MB restriction;
gcc
consists of one large
load module where all calls are simple static calls (using V
constants).
Whether GCC of this vintage is one large module (which I doubt very
much) or not, I do not know. The reason I say that is because the
discussion at the time suggested one module tripped this constraint.
A solution though was found. Increase the amount of available memory.
This was the solution chosen by the original developers behind the GCC
porting to MVS and VM. The solution involved allowing a S/370
operating system to run with 31-bit addressing. This was the inception
of the S/380 version of Hercules and alterations that allowed GCC (or
any other application) to use the increased memory. That group has now
split off entirely from Hercules. At this point, I ceased tracking
what they are doing.
I know the underlying Hercules address-mode change was never included
in Hercules. Whether the current versions in use on Hercules of MVS
and VM/370 include the required changes I do not know. Someone closer
to those operating systems might know the status. I think there were
separate versions of MVS and VM that had changes that used the Hercules
31-bit address mode in S/370, that is, a MVS/380 and a VM/380.
I could imagine a much better solution, but improving this requires
a
deep knowledge of the
overall gcc structure and a decision, which parts of gcc could maybe
be
transformed into
seperately loadable "phases". If the gcc compiler does not have
parts
that run sort of
separately from the rest and can be separated using LOAD / DELETE or
LINK macros,
then IMO all bets are off.
Of course: gcc is written in C, but anyway:
even C functions can be loaded (and removed) dynamically using the
MVS
macros mentioned above;
I've done this in the past using IBMs C with C function pointers (by
writing ASSEMBLER functions
cload() and cdelete() usable from C, returning C function pointers).
This is possible.
But the hard part IMO is:
to identify the parts of gcc where such a change makes sense
and where it would help to reduce the overall gcc load module size.
Nobody was willing to tackle this problem with the i370 version of gcc.
A function which is cload()ed from the rest of gcc must be linked
together with all of its needed subfunctions,
and in the best case it should not need subfunctions used in the rest
of
gcc ... otherwise the needed
space for these subfunctions will double.
That is:
all the functions of gcc must be separated in several different sets
of
functions, which are complete
(that is, don't need functions from another set) AND the different
sets
can be cload()ed step be step
from a main section at different times. If this is accomplished,
then
the overall size used will be
only the size of the largest set plus the size of the main section
(and
not the sum of all sections,
as today).
I don't know if this is possible. It is probably needed to analyse
all
the function calls first.
HTH, kind regards
Bernd
Am 28.10.2022 um 17:51 schrieb rvjansen@...:
Just a wild question here with regard to the compiler we need to
use for cREXX.
The gcc compiler ported to VM/370 suffers address space size
problems (so I have heard from reliable sources) and even has led
to the ¡®380¡¯ version of Hercules to be able to be compiled (so I
have read). A larger language like PL/I does not have these
problems however, and when I read up about its history, this seems
to be due to the fact that it has a lot of ¡®phases¡¯ that could be
implemented in overlays, something the linkage editor/loader could
handle in the pre-virtual memory days (and following IBM¡¯s
reputation for preserving compatibility, probably still can).
This has led to the anecdote that one IBM lab was toiling at phases
and overlays to fit PL/I in even the smallest of S/360 machines,
while the other team was putting the finishing touches on virtual
memory. Should they have talked to each other, a lot of work could
have been avoided.
Now gcc is a lot bigger than anything from that era, and we have
problems - where it first compiled cRexx on VM/370, now it does
not. My speculation is that if we went in the opposite route than
the historic development went, and we packaged this compiler in
overlays, we could lessen its memory footprint.
Is there anyone from that era who still knows how this is done, can
tell us whether it would be possible, and can advise on what to do?
A bigger issue would be someone with the experience and understanding
of how GCC functions to do this work. That is where the rub is. A
number of people here have probably done overlay work. It is the
knowledge of C plus compiler design that is the real challenge for this
group.
For the rare occurrences that require more than 16M to compile C code,
the "compile on a PC" approach would be recommended by me. Of course
getting access to the i370 GCC compiler that runs on a PC is the
challenge. Someone who still works with the "380" team may be able to
ascertain that.
As someone who has actually implemented an assembler, no where near as
complicated as a compiler, this is not trivial.
And I would certainly not touch it at all if C was a requirement.
best regards,
¸é±ð²Ô¨¦.
