Inside VESOFT #06 (Expression Programming)
by
Shawn M. Gordon
One of the more fascinating aspects of the VESOFT products is the expression programs that implement almost all of the functionality of the product. The Appendix of the manual goes on in pretty good detail on how to use expression programs for command files and custom LISTF modes, so I won’t go into a huge amount of detail in this column.
How about a little history on it first. Eugene set about rewriting MPEX some 12 or so years ago, and as legend has it, he ended up writing a general purpose compiler. The language looks a lot like Pascal in places, but over the years has implemented some bits from the C language as well. It’s like an object oriented, arbitrary expression parser. It is really incredibly powerful and versatile, and almost completely undocumented. If you go and look at any of the command files in CMD22.VESOFT that have a file code of 171, you will notice all sorts of nifty commands, and then when you try to say HELP command, nothing will come up.
VESOFT only documents the things that they are going to basically guarantee will continue to work. They reserve the right to change anything that is undocumented as much as they want. So while you might be able to figure out how to use something, it is not suggested that you rely on it to continue to work as time goes by. I think this is perfectly acceptable, and is clearly described in the manual.
Now the functions that you see inside the command files in CMD22.VESOFT are implemented in the AA files in EXP22.VESOFT. When I first looked at these, I assumed they were encrypted somehow, but it turns out that they aren’t, this is what they are supposed to look like. Here is a short example:
~0 [* SEC ~C ~V 70 DEL=, TERM=] SEC
:iiD iINDEXNB
:bbD iINDEXNB
:ddD |kkD |rrD |ttD dINDEX
~0 [^ ~C ~V 70 TERM=] DEL=: SEC
:SiDD ISUBSTR
:SSDD SSUBSTRFIX
~0 ASL ~C ~V 60 :DDD DASL
~0 ASR ~C ~V 60 :DDD DASR
Now both the expression programs, and the AA files get pseudo-compiled into other files with a file code of 176 and 8 appended to the name (this is different under CM). When the VESOFT software is installed they go through and precompile as much as possible. Basically these things get dynamically compiled when you use them if they have changed since the last compilation. It’s very convenient, but also means that the first run is a bit slower. This will pertain more often to your own expression programs.
What is really interesting is that all the VESOFT products use the same engine in the background, but their expression programs all go into different groups. It was really an elegant solution for a pretty large and complex question. On the surface, there is very little related to MPEX, Security and VEAUDIT. It takes some serious abstraction to follow the motivation, and once you get there, it makes perfect sense.
Ok, I’ve gone on long enough about the background, how about a little sample.
Many moons ago when I worked at VESOFT, one of the other tech support guys wanted to know what was in an SD (Self Describing) save file from Query. I happened to remember reading an article on it in the SCRUG Newsletter about a year before. Since no publication ever gets thrown away at VESOFT, we browsed till we found it. I decided to use the information to write an expression program that would read the file label of an SD file and extract the data and format it.
See Figure 1 for the code example.
Much of this should be obvious, even if you don’t know Pascal. Expression programs don’t support record structures like COBOL, so you have to parse out the data based on position. I’m also using an unsupported construct, the SUBROUTINE directive (don’t use it at home). VEPARMS is a predefined variable that will contain what is passed on the command line. In this case it is a simple file name with no parameters, so we can just use it to get the file name and the file label information.
The funny thing about SD files is that the data is stored in the file label in reverse order, so you have to read through it backwards. I don’t have the original article anymore, so I always have to read through this expression program to figure out what went where. I don’t remember what is in the first 8 records anymore, or the last two. I’ll leave it as an exercise for the reader.
Figure 2 shows the original command file, and the compiled file that was generated by MPEX. Figure 3 shows an example of the output from executing the expression program.
I hope you’ve enjoyed this little history lesson, and tiny forray into expression programming. I’ll cover it more in the future for various types of uses. Make sure to send in your tips too.
Figure 1
PRINT SDREAD.CMD
VAR SD_LABEL :STRING[256];
VAR FULL_NAME :STRING[26];
VAR OUT_TYPE :STRING[1];
VAR ITEM_NAME :STRING[16];
VAR ITEM_TYPE :SHORTINT;
VAR ITEM_OFFSET :SHORTINT;
VAR ITEM_LENGTH :SHORTINT;
VAR FNUM :INTEGER;
VAR NUM_LABEL :INTEGER;
VAR LOOP :INTEGER;
VAR S :INTEGER;
VAR I :INTEGER;
SUBROUTINE SETTYPE;
BEGIN
if ITEM_TYPE = 1 then OUT_TYPE := 'X';
if ITEM_TYPE = 3 then OUT_TYPE := 'I';
if ITEM_TYPE = 4 then OUT_TYPE := 'R';
if ITEM_TYPE = 5 then OUT_TYPE := 'P';
if ITEM_TYPE = 6 then OUT_TYPE := 'J';
if ITEM_TYPE = 7 then OUT_TYPE := 'K';
if ITEM_TYPE = 8 then OUT_TYPE := 'Z';
if ITEM_TYPE = 10 then OUT_TYPE := 'C';
END;
FULL_NAME := vefinfo(VEPARMS).fullname;
FNUM := vefopen(FULL_NAME+',OLD;ACC=IN');
NUM_LABEL := vefinfo(FULL_NAME).labeleof;
LOOP := NUM_LABEL - 2;
writeln('Data Item':18, 'Image':9, 'Length':10, 'Starts':6);
writeln(' Type in Bytes at Pos');
writeln(43*'-');
while LOOP > 9 do begin
SD_LABEL := vefreadlabel(FNUM,LOOP);
I := 0;
S := 0;
(* writeln('Label: ', LOOP, ' ', SD_LABEL:'GARBAGE'); *)
while I < 8 do begin
ITEM_NAME := SD_LABEL[S:15];
ITEM_TYPE := INTEGER(SD_LABEL[(S+16):2]);
ITEM_OFFSET := INTEGER(SD_LABEL[(S+18):2]);
ITEM_LENGTH := INTEGER(SD_LABEL[(S+20):2]);
SETTYPE;
writeln(ITEM_NAME:20,OUT_TYPE:4,ITEM_LENGTH:6,ITEM_OFFSET+1:10);
I := I + 1;
S := S + 30;
if I < 8 then ITEM_TYPE := INTEGER(SD_LABEL[(S+16):2]);
if ITEM_TYPE < 1 then I := 10;
end;
LOOP := LOOP - 1;
end;
Vefclose(FNUM);
Figure 2
ACCOUNT= SMGA GROUP= CMD
FILENAME CODE ------------LOGICAL RECORD----------- ----SPACE---- --DAYS--
SIZE TYP EOF LIMIT R/B SECTORS #X MX ACC MOD
SDREAD 171 72B FA 53 53 16 16 1 5 59
SDREAD8 176 128W FB 10 1023 1 16 1 *
GROUP TOTAL: 2 FILES 32 SECTORS
Figure 3
%SDREAD X2
Data Item Image Length Starts
Type in Bytes at Pos
-------------------------------------------
MAIL-NAME X 12 1
TERMINAL-NO J 2 13
PRINTER X 8 15
PHONE-EXT J 2 23
DEPT X 26 25
NODE X 8 51
FLAGS X 16 59
USER-PASS J 2 75
USER-NAME X 30 77
TIME-ON J 4 107
DATE-ON J 2 111
FLAGS2 X 80 113