File: /usr2/fs/misc/fs10.0.0up.txt Version: 10.0 Date: 200420 FS 10.0.0 Beta Update Notice I. Introduction This document is divided into five sections: I. Introduction II. Installation as an upgrade to a previous version III. Converting to a 64-bit system IV. Changes from previous versions A. Reference Appendix for Installation This update is intended for all stations using the FS. This includes both stations using the main (non-VGOS) branch (latest version 9.13.2) and the VGOS branch (latest versions 9.12.10-9.12.13). Installing the new version as update in either case is fairly simple. The most significant changes in 10.0.0 are: 1. One unified version for all FS users Both non-VGOS and VGOS operations are supported by the new version. Previous users of the main (non-VGOS) branch will need to add some new local control files. The most significant change is that previous main branch users will find that in the new FS input is case sensitive, as it was already in the VGOS branch. 2. Support for use on both 32- and 64-bit systems. The new version should run on existing 32-bit systems that currently support the FS, as well as on 64-bit systems. The update notice includes instructions for transitioning to 64-bit systems, which may require some relatively simple changes to station software. A new FS Linux distribution, FSL10, is now available or will be shortly, for 64-bit (and 32-bit) use. We recommend using this distribution if you would like to move to using a 64-bit OS or need a 32-bit OS that is still under support (FSL9 is no longer under support). Release of FSL10 will occur separately. 3. Version control with 'git'. The FS source is now managed using 'git' and 'github' is used as the distribution method. The details of the model used for FS releases under 'git' are described in /usr2/fs/misc/release_model.txt. Very significant contributions, in fact essential, were made by Dave Horsley (UTAS, Australia and GSFC) and Tetsuro Kondo (NICT, Japan and SHAO, China). Kondo-san pioneered 64-bit support in the FS with his 64-bit implementations for Sejong and Ishioka stations. Dave built on his work in developing a version that would support 32- and 64-bit systems. Dave also single-handedly imported the entire existent history of FS source code into 'git'. This includes over 130 FS9 versions (under Linux), 17 FS8 versions (under VENIX), and two older versions (under HP RTE-1000/A) going back to version 5.5 in 1988. Having the source code in 'git' greatly simplified the task of merging the non-VGOS and VGOS versions. Dave also did the most critical work on that and provided excellent guidance on using 'git'. Having the historical code in 'git' is a great resource for understanding the evolution of the code. As always, we are deeply indebted to Jonathan Quick (HartRAO, South Africa) for his many significant contributions that go far beyond what is explicitly mentioned below. In this particular case, this includes testing and insights for adapting the code for 64-bit use, patiently solving various problems, developing FSL10, and working out support for older FSL distributions, particularly FSL8. II. Installation as an upgrade to a previous version Since this section is written as upgrade to an existing installation, it is assumed that you are using a 32-bit system. However, only a few things here are different for 64-bit systems. As result this section is also in the instructions for converting to a 64-bit system, with appropriate embedded information along the way. Please see section III 'Converting to a 64-bit system', for overall instructions for converting to 64-bit. An appendix A. 'Reference Appendix for Installation' is provided at the end of this document. It includes seven sub-sections: A.1. Upgrading from FS versions before the previous stable A.2. Example standard procedure libraries A.3. Cut-and-paste installation tips A.4. Making a back-up before installing A.5. Disk space requirements A.6. Set operations file permissions A.7. Fix '.prc' file 'define' lines If you haven't upgraded or installed the FS before, you may want to review the appendix. It is STRONGLY recommended that you back-up your operational system before upgrading. There are two possible paths for upgrading: from a main branch version or from a VGOS branch version. There are some general considerations that depend on which branch you are updating from: 1. Upgrading from a main branch version. The main branch versions are numbered 9.13.x and 9.11.x or older. Specifically, versions 9.12.x are not part of the main branch. If you are upgrading from a main branch version, it is assumed that upgrade is from 9.13.2, the previous stable release. If you have a main branch version older than version 9.13.2 you should upgrade to 9.13.2 first, please refer to appendix sub-section A.1. 'Upgrading from FS versions before the previous stable', for more information. 2. Upgrading from a VGOS branch version. The VGOS branch versions are numbered 9.12.x. The instructions provided in this section, below, are for installing as an upgrade to versions 9.12.10-9.12.13, the latest VGOS branch releases. As far as we know, no other VGOS versions are in use. If you have a different version, please contact Ed (Ed.Himwich@nviinc.com). The upgrade instructions for the two paths differ only in step (12) below. The installation steps for upgrading are: (0) Back-up your operational system. Having a back-up to return to will allow you to continue operations in case something goes wrong with the installation. For more details, please see appendix sub-section A.4 'Making a back-up before installing'. NOTE: If you are using FSL10, that sub-section points you to the improved backup and test procedure that is available with that distribution. NOTE: That sub-section also includes a description of how to preserve your operational files and switch back and forth between an operational and a test set-up by changing symbolic links. (1) Login as 'root'. (2) Place a copy of the FS 'git' repository in the '/usr2' directory on your computer For example, you might do the following: cd /usr2 git clone https://github.com/nvi-inc/fs.git fs-git or, alternatively, if you need to use 'ssh' instead: cd /usr2 git clone git@github.com:nvi-inc/fs fs-git (3) Checkout the beta release from the local repository. cd fs-git git checkout 10.0.0-beta1 (4) Set the link for the new FS version: cd /usr2/fs-git make install (answer 'y' to confirm installation) NOTE: This step will change your '/usr2/fs' symbolic to point to '/usr2/fs-git'. You will need to change the link manually to switch back to your old version. The 'make install' step may create and possibly rename some existing directories if the FS was never installed on this system before. However, if you are using this step as part of upgrading on your 32-bit system (this step is not used on a 64-bit system you may be converting to), this should not be an issue. (5) Having the wrong ownership and/or permissions on the operational files (procedure libraries, control files, schedules, and logs) can cause errors during FS operations. For a full discussion, please refer to sub-section: A.6. 'Set operations file permissions'. For stations where all the operational files are expected to owned by user 'oper' in group 'rtx', with permissions ('ug+rw,o+r,o-w') the following command will enforce this (note that the "execute/search" bits are not changed): /usr2/fs/misc/fix_perm Answer 'y' to the prompt if you wish to proceed. (6) ** VERY IMPORTANT ** Log-out as 'root', and log-in as 'prog'. ** VERY IMPORTANT ** (7) Make the FS itself: A. Set the compiler. Starting with version 10.0.0, the standard FORTRAN compiler for use with the FS is 'f95' and we recommend that you use it. On the 32-bit systems you can still use 'fort77', but you should only use it if you either you don't have 'f95' or if you have any FORTRAN station code that is too difficult to convert to 'f95', see step (8) for more details. To select 'f95' as your compiler, you will need to set the 'FC' variable to this value. If your shell is 'tcsh' you can use: setenv FC f95 If your shell is 'bash', you can use: export FC=f95 NOTE: For beta testing on a 32-bit system, you may not want to make this change permanent since it is incompatible with pre-10.0.0 versions. To make this change permanent, you should add the appropriate command to the appropriate "rc" file depending on your login shell: '~prog/.login' for 'tcsh' or probably '~prog/.profile' for 'bash'. B. Make the FS: cd /usr2/fs make >& /dev/null and then make -s to confirm that everything compiled correctly (no news is good news). (8) This step is for modifying your station programs, in '/usr2/st'. There are two possible issues: (sub-step A) the impact of FS input being case sensitive on the 'antenna=...' command, and (B) conversion of FORTRAN code. Even if neither of these issues affect you, you must recompile and relink your programs against the new FS version in any event, as described in step (9). A. If your antenna, or your side of the antenna interface, requires that the strings passed by the 'antenna=...' command are uppercase, you have two options: i. Convert your code. For simple backward compatibility, change you 'antcn' program to always convert the 'antenna=...' strings to upper case. Alternatively, make your code case insensitive. ii. Convert the strings in your 'antenna=...' commands wherever they occur: SNAP procedures, SNAP schedules, external programs, or scripts, to upper case. The former choice (A) is probably the easier, but in some cases (B) may be better. If you have question about which to use and how to do it, please contact Ed (Ed.Himwich@nviinc.com). B. Conversion of FORTRAN code. If you have station programs in FORTRAN, please contact Ed (Ed.Himwich@nviinc.com), but basically you have two options: i. Use 'f95' for both the FS, see step (7), and your station FORTRAN programs. You will need to adapt your 'Makefiles' to use the same compiler options as the FS, which can be found in '/usr2/fs/include.mk'. It is recommended that you follow this approach for 32-bit systems and it is necessary when moving to a 64-bit system. As a first cut, it may work to add the following two lines to your 'Makefile's for FORTRAN programs: FFLAGS += -ff2c -I../../fs/include -fno-range-check -finit-local-zero -fno-automatic -fbackslash FLIBS += -lgfortran -lm ii. Continue to use 'fort77' as the compiler for both the FS, see step (7), and your station programs. You should follow this approach ONLY if you are on a 32-bit system and it is too difficult to convert to 'f95'. When done, be sure to recompile and relink your programs against the new FS version in any event, as described in step (9). (9) Remake your local software. If '/usr2/st/Makefile' is set-up in the standard way, you can do this with: cd /usr2/st make rmdoto rmexe all At this point, you are only trying to verify the code will 'make' successfully. You may still need to debug it in the step (16) below. (10) ** VERY IMPORTANT ** Reboot the computer. This is important for initializing shared memory for the new version. ** VERY IMPORTANT ** (11) Log-in as 'oper'. (12) This step is for updates to the local control files. There are four changes that are needed, listed here as steps A-D. Differences for updating from different versions are noted. Please read each step, including sub-steps, carefully to make sure you find all the clauses for your old version; sometimes an old version is included in more than one clause. A. The 'stcmd.ctl' file needs another digit added to the subroutine number. This step is only need for updates from 9.13.2. You can fix your file with the commands: cd /usr2/control /usr2/fs/misc/cmdctlfix6 stcmd.ctl You may also want to expand the (typically) second comment line to correspond to the new format by adding a 'U' after character 18 to read as: *COMMAND SEG SUBPA BO B. You will need to execute the following commands to copy new files that are needed (cut-and-paste is your friend). There are three cases depending on what your old version was: i. If your old version was 9.12.10-9.12.12: cd /usr2/control cp /usr2/st.default/control/clpgm.ctl . cp /usr2/st.default/control/rdbemsg.ctl . ii. If your old version was 9.12.13: cd /usr2/control cp /usr2/st.default/control/rdbemsg.ctl . iii. If your old version was 9.13.2: cd /usr2/control cp /usr2/st.default/control/dbba2.ctl . cp /usr2/st.default/control/mk6c?.ctl . cp /usr2/st.default/control/monit6.ctl . cp /usr2/st.default/control/rdbc?.ctl . cp /usr2/st.default/control/rdbe.ctl . cp /usr2/st.default/control/rdbemsg.ctl . C. Update your 'equip.ctl' file for the DBBC3 configuration line. There are two cases, please check which, if either, applies for you. i. If your old version was 9.12.10-9.12.13, you will need to add the final four lines of the example 'equip.ctl' file to yours: cd /usr2/control tail -n 4 /usr2/fs/st.default/control/equip.ctl >>equip.ctl ii. If your old version was 9.13.2, you will need to add the final two lines of the example 'equip.ctl' file to yours: cd /usr2/control tail -n 2 /usr2/fs/st.default/control/equip.ctl >>equip.ctl D. You should compare your versions of the following files: clpgm.ctl equip.ctl stpgm.ctl to the examples, e.g.: cd /usr2/control diff clpgm.ctl /usr2/fs/st.default/control/ | less and consider whether you should make any changes to your copies. In the case of 'clpgm.ctl', you might just be able to replace your copy with the new one. The following is a list of changes in these files: clpgm.ctl - This file was not present for 9.12.10-9.12.12, so if your old version was one of those, the new default version (copied by commands in step B above) should not require modification. - If your old version was 9.12.13 or 9.13.2: the '-title ...' parameter for each window was removed so that it is uniquely supplied by the '.Xresources' file. - If your old version was 9.12.13 or 9.13.2: new lines were added for the useful display window 'scnch', and the useful RDBE display windows: 'monit6', and 'monX' ('X'=[a-b]). The 'monan' program was added since it is used at several sites. equip.ctl - This file has the most complicated changes. Please read all clauses to make sure you see all that apply to your old version. The first section covers changes to non-comment lines; the second, comments. The former are essential. The later are in some sense optional, especially when they refer to equipment you don't (or never will) have. However, changing them now may help avoid confusion at a later date. Non-comment lines - If your old version was 9.12.10-9.12.13, the line for DBBC PFB version should be changed to (shown here with the typical preceding comment): *DBBC PFB version v15_1 v15_1 or later - If your old FS version was 9.12.10-9.12.13, the line that defines the DBBC2 CoMo configuration has changed. Please see item (12) in the installation instructions in 'fs91119up.txt' for full details on handling this. However, the following commands will probably work if you don't have a DBBC2 or if your DBBC2 configuration is four CoMos with one Core per CoMo: cd /usr2/control /usr2/fs/misc/dbbc_equip '1 1 1 1' equip.ctl If the script prints a warning about the number of IF power conversions being incorrect, the issue must be resolved before continuing, either by adjusting the number of power conversions, adjusting the CoMo configuration, or both. - If your old version was 9.12.10-9.12.12 or 9.13.2, a new line for the DBBC3 configuration was added at the end, but step C above should have handled that. Comment lines - Compared to all old versions, comment lines were added or modified for new equipment type options. - If your old version was 9.12.10-9.12.13, a comment line describing the met sensor was changed. - If your old version was 9.12.10-9.12.13, the comment lines describing the available clock rates was completely rewritten and greatly expanded, and an additional clock rate (128) was appended to the end of the comment on clock rate line itself. stpgm.ctl - New lines were added for 'erchk', 'monit2', and 'scnch' to be started by 'fsclient' when the display server is in use. The new 'erchk' line differs from the previous commented version in that the '-title ...' parameter has been removed so that it is uniquely supplied by the '.Xresources' file. (13) This step is for updates to your SNAP '.prc' procedure libraries. Only one change is needed, to convert use of the old FS 'go' program to use 'rte_go'. This change came about because the compiler for the 'go' language conflicts with the old name. This change is necessary even if you do not have the 'go' language compiler installed. To make this change for all your '.prc' procedure libraries, execute: cd /usr2/proc /usr2/fs/misc/go_fix *.prc Files that are changed will have a pre-change back-up copy with the extension '.bak'. (14) This step is for miscellaneous FS related changes. There are are none for this update. (15) This step is for miscellaneous FSLx changes. None are required for this update. (16) Test the FS as 'oper'. NOTE: If you are following the process in this section on your 32-bit system as part of converting to 64-bit (section III below), you may prefer to save extensive testing until you are verifying the 64-bit installation. You could always go back and do more testing on the 32-bit system to help resolve the origin of problems that are noticed on the 64-bit system. Generally speaking, a fairly thorough test is to run a test experiment. Start with using 'drudg' to rotate a schedule, 'drudg'-ing it to make '.snp' and '.prc' files, making listings, and any other pre-experiment preparation and tests you normally do, then execute part of schedule, and perform any normal post-experiment plotting and clean-up that you do. The idea here is to verify that everything works as you expect for normal operations. (17) Consider when to update your back-ups. NOTE: This step may not be appropriate if you are beta testing since the beta test versions are not intended for operations. It would be prudent to wait until you have successfully run an experiment or two and preferably received word that the experiment(s) produced good data. The chances of needing to use your back-up should be small. If something does happen, you can copy the back-up to the (now assumed bad) updated disk. You can then either use the restored disk or apply the FS update again. The FSL10 test procedure has more options for recovery. Managing this is a lot safer if you have a third disk. (18) We encourage you to enable the display server interface of the FS if you aren't already using it. Instructions for enabling this feature (and disabling it, if that should become necessary) can be found in '/usr2/fs/misc/display_server.txt'. This file also includes a explanation of how the user interface is different from the non-display server. III. Converting to a 64-bit system To upgrade your installation to 64-bit, the easiest approach is probably to first upgrade the FS installation on your 32-bit system to run FS 10.0, then transfer it to a 64-bit system. This will allow you verify the upgrade before trying to transfer it. An alternate method is to copy the station files from your 32-bit system and then perform the upgrade. This will allow you to upgrade without modifying your existing system. These two methods only differ in when the files are copied to the new machine, in step (3), and when the update instructions in section II above are used, step (3) or step (5). The instructions below assume that the existing system you are using has a standard FS configuration in terms of symbolic links and directories. If your system is different, you will need to adjust what you do accordingly, but you may still find the outline of steps useful. Please follow these steps: (1) Install a 64-bit system on a different (hopefully new) computer according to: https://nvi-inc.github.io/fsl10/ NOTE: Despite what the URL above says about which version of the FS to check-out, at this time you should check-out '10.0.0-beta1', You can install a different distribution. However, we can't provide as much support in that case. The FSL10 distribution is tuned to provide a complete platform for running the FS. For other distributions you may have to make adjusts for several things that an FSL10 installation provides, including: - which packages are installed - required user accounts and groups (2) Rename the existing default station specific FS related directories on the 64-bit computer to get them out of the way, e.g.: cd /usr2 mv control control.DEFAULT mv sched sched.DEFAULT mv proc proc.DEFAULT If you use the '/usr2/tle_files' directory on your old computer, you should rename it on the new computer: cd /usr2 mv tle_files tle_files.DEFAULT If your station software is in '/usr2/st-0.0.0' on your old computer, you should rename in on the new computer: cd /usr2 mv st-0.0.0 st-0.0.0.DEFAULT The 'log' directory were not included above, since it should essentially be empty on the new computer, but could also be renamed if you prefer: cd /usr2 mv log log.DEFAULT (3) For copying your files, there are two options: A. If you are upgrading your installation on your old computer first, please follow the directions in section II above for your old system. Then return here and follow the steps in "Copying your files" below, then proceed to step (4) below. B. If you are copying your files first, follow the steps in "Copying your files" below, then proceed to step (4) below. Copying your files Your station specific FS related files can be copied from the old computer using any convenient method. If both systems are on the network this can be particularly easy. The following example steps use this approach. You should replace the example hostname 'old' with your old systems hostname. You may need to provide the appropriate password for each 'scp' command (if so, you can simplify the process by coping your 'root' 'ssh' keys to the new machine with 'ssh-copy-id' first). These commands must be executed as 'root': i. Transfer operations directories: cd /usr2 scp -pqr oper@old:/usr2/control . scp -pqr oper@old:/usr2/sched . scp -pqr oper@old:/usr2/proc . Note that your 'sched' and 'proc' directories could be large and take a significant amount of time to transfer. It can be useful to have your old log files on the new computer, but that transfer could be even larger: cd /usr2 scp -pqr oper@old:/usr2/log . If you use the '/usr2/tle_files' directory on your old computer, you can also transfer it: cd /usr2 scp -pqr oper@old:/usr2/tle_files . ii. Fix the permissions on the operations directories/files you transferred. You can fix their permissions and ownerships to the standard with: /usr2/fs/misc/fix_perm Answer 'y' to confirm. If you don't have a '/usr2/tle_files' directory, you will get a message that there is no such directory. That is benign unless you expect such a directory to be there. iii. Make back-up copies of the operations directories. This step is optional but may be useful so there are unmodified copies of the directories from the old machine to use for reference: cd /usr2 cp -a proc proc-old cp -a control control-old cp -a sched sched-old and possibly: cd /usr2 cp -a log log-old cp -a tle_files tle_files-old iv. Transfer your station software directory (and make a reference copy). This is usually the target directory pointed to by the '/usr2/st' symbolic link. On your old computer, you can find its name with: ls -l /usr2/st In the rest of this step, the target 'st-1.0.0' will be used as an example, but you should replace it with your actual target. (If your target is 'st-0.0.0' you should rename the default on the new computer as described in step (2) above). On the new computer, copy the target from the old computer to the new computer, e.g.: cd /usr2 scp -pqr oper@old:/usr2/st-1.0.0 . On the new computer, set the '/usr2/st' symbolic link to point to the target directory: cd /usr2 ln -fsn st-1.0.0 st You can set its permissions and ownership for 'prog' with: cd /usr2 chown -R prog.rtx st-1.0.0 chmod -R ug+rw,o+r,o-w st-1.0.0 You can make a reference copy with: cd /usr2 cp -a st-1.0.0 st-1.0.0-old v. Copy your 'oper' and 'prog' directories to the new computer. This step is optional. The FSL10 installation made default home directories for these users on '/usr2'. If you did not have customized content for the users on the old computer, you could just use the versions on the new computer. Still it may be useful to have a copy of your old directories on the new system for reference, especially if you realize later that there were additional programs and files you want to use on the new system. You can accomplish the transfers as 'root' using: cd /usr2 scp -pqr oper@old:~ oper-old scp -pqr prog@old:~ prog-old You may want set their permission and ownership so the appropriate user can access them: chown -R oper.rtx /usr2/oper-old chmod -R a+r,u+w,go-w /usr2/oper-old chown -R prog.rtx /usr2/prog-old chmod -R a+r,u+w,go-w /usr2/prog-old You can customize the home directories on the new computer to include any features you want from the old system. vi. At this point you are principle done transferring files. However, it is also possible that you may need or want other changes such as: . Copy other files or programs from the old system . Install additional Debian packages . Copy/set-up additional configuration files, such as: /etc/hosts /etc/hosts.allow /etc/hosts.deny /etc/ntp.conf You can use a similar process to the one above to transfer and/or make reference copies of more files and directories. (4) Converting your station code. There is one general issue, (sub-step A) handling string passed by the 'antenna=...' command. Then you may need to use one or both of the additional steps below depending on whether you have (B) FORTRAN and/or (C) C station code. You may have already handled issues (A) and (B), if you updated on a 32-bit system first (for FORTRAN if you did not convert to using 'f95', you will still need to do that). In any event, make sure your code 'make's successfully before proceeding to step (5). You may still need to debug it later. A. If you pass strings to your antenna, or your side of the antenna interface, with 'antenna=...' commands, you may need to change how those strings are handled. If you have not already done this, please see step (II.8.A) above for the details. B. If you have FORTRAN station code, it will need to be converted to use 'f95'. If you have not already done so, please see step (II.8.B) above for the details. Please contact Ed (Ed.Himwich@nviinc.com) if you have FORTRAN station code, regardless of whether you have a problem converting it or not. C. If you have C station code, it should work as written unless you have declared integers that interface to the FS as 'long'. For a start at fixing this, please see the URL: https://github.com/dehorsley/unlongify A small amount of set-up is needed to use the 'unlongify' tool. If you are using FSL10, you can install the 'go' language in one of two ways listed below. We recommend the first way for those that are only using 'go' for the 'unlongify' tool. i. You can use the Debian package management system to install 'go'. This will give you an older version of 'go' that is perfectly adequate for the task at hand and is supported by the normal security mechanism. To install it this way, as 'root' use: apt-get install golang ii. You can install the latest version of 'go', but this is outside the normal security mechanism. In this case, you will need to manage your own updates, which may not be suitable for an operational environment. Both the initial install and updates are handled by the 'fsadapt' script, as 'root': cd /root/fsl10 ./fsadapt In the first window select *only* the option: 'Install' (or 'Update') 'Go programming language' Then press 'Enter' on *OK*. On the next screen press 'Tab' to highlight 'Cancel' and press 'Enter'. Once you have the 'go' language installed, you need to define the 'GOPATH' environment variable and include it in 'prog's path. The default '~prog/.profile' file includes two commands (commented out by default) to accomplish these things: #export GOPATH=~/go #PATH="$GOPATH/bin:/usr/local/bin/go:$PATH" You will need to uncomment these two lines and then log-out and log back in again as 'prog' or, in a current login session for 'prog', re-execute the file: . ~/.profile Then you should be able to execute the installation step given at the URL above (as 'prog'): go get github.com/dehorsley/unlongify Please read the 'README.md' file, which is displayed at the URL above. Alternatively, it can be viewed at ~/prog/go/src/github.com/dehorsley/unlongify/README.md' where it was installed by the above command. Please pay particular attention to the 'Note' about system calls. Once your code 'make's successfully proceed to step (5). You may still need to debug it. (5) If you upgraded to FS10 before copying your files and have made any needed changes in your station code as described in step (4) above, you should re-test your system, in fact following all the steps in section II starting with step (16). If you copied your files before upgrading to FS10, and have made any needed changes in your station code as described in step (4) above, you should complete the upgrade by starting with step (10) in section II and completing it and all the following steps. IV. Changes from previous versions This section is divided into four sub-sections: A. FS Changes compared to 9.13.2 B. FS Changes compered to VGOS versions C. Changes in 'drudg' D. Known FS Bugs Each sub-section starts with a summary of the items covered followed by a more detailed description. A. FS Changes compared to 9.13.2 The following is a summary of FS changes since 9.13.2. *TO BE WRITTEN* 1. Source version control is maintained with 'git'. 2. Source code now works on 32- and 64-bit platforms. 3. Input is now case sensitive. A more detailed discussion of these changes follows. *TO BE WRITTEN* 1. Source version control is maintained with 'git'. 2. Source code now works on 32- and 64-bit platforms. 3. Input is now case sensitive. A complete history of the routines changed can be found using the 'git log' command. B. FS Changes compared to VGOS versions The following is a summary of FS changes compared to VGOS versions. *TO BE WRITTEN* 1. Source version control is maintained with 'git'. 2. Source code now works on 32- and 64-bit platforms. A more detailed discussion of these changes follows. *TO BE WRITTEN* 1. Source version control is maintained with 'git'. 2. Source code now works on 32- and 64-bit platforms. A complete history of the routines changed can be found using the 'git log' command. C. Changes in 'drudg' *TO BE WRITTEN* 'drudg' opening message date is 2019Nov20. The following is a summary of 'drudg' changes *TO BE WRITTEN* 1. Source version control is maintained with 'git'. 2. Source code now works on 32- and 64-bit platforms. A more detailed discussion of these changes follows. *TO BE WRITTEN* 1. Source version control is maintained with 'git'. 2. Source code now works on 32- and 64-bit platforms. A complete history of the various changes and the routines they affect is in /usr2/fs/drudg/change_log.txt. D. Known FS Bugs (There have been no changes in this section since the previous version.) The following is a summary list of known bugs. They are described in more detail after the list. 1. Do not run 'fmset' for extended periods. 2. 'odd' and 'even' head types not supported for Mark IV & VLBA4. 3. 'odd'/'even' head types not supported for VLBA style tapeforms. 4. 'chekr' does not check the status of the Mark IV formatter or Mark 5 recorder. 5. Extraneous errors when tape is stopped by low tape sensor. 6. 'Comm=' command in logex extracts only the first command. 7. S2 error scheme clumsy. 8. No extra spaces allowed in 'ibad.ctl' file. 9. ONOFF and FIVPT programs hang. 10. FS SNAP command pages don't list tape drive suffix numbers. 11. LBA rack TPI detector is not usable. 12. mk5b_mode and bit_stream commands only report the expected sample rate. 13. Some fmpsee routines do not report file I/O error through the log system. 14. Some systems calls, particularly in mk5cn and dbbcn, use separate UN errors to elaborate on errors in system calls. A more detailed discussion of these bugs follows. 1. Do not run 'fmset' for extended periods. For stations that have formatter that can be set with 'fmset', the program should not be run for extended periods of time. The 'fmset' program should be used only to set or briefly verify that the formatter time is correct. Do not leave 'fmset' running after completing either of these tasks, especially during an experiment. The 'fmset' program dominates the Field System when it is running and this is likely to interfere with the running of an experiment or other activities. The only way to detect the time from the VLBA formatter with greater precision than one second it to wait for the seconds response from the formatter to change. This requires the FS to communicate with the formatter almost continuously. A similar problem exists for the S2 recorder. This problem is less severe for other formatters, but extended use of 'fmset' in this case should be avoided as well. A reminder about this is included in the 'fmset' menu. 2. 'odd'/'even' head types not supported for Mark IV & VLBA4. The Mark IV and VLBA4 rack version of the 'form' command and the Mark IV and VLBA4 recorder versions of the 'repro' and 'parity' commands do not support the 'odd' and 'even' parameters for the read and write head types and reproduce electronics in the 'head.ctl' control file. This means that automatic substitution of odd or even head in passes that use only even or odd heads respectively does not occur. The only correct settings for the read and write head parameters and reproduce electronics is 'all'. This will be fixed in a future revision. Please let Ed know if you are missing some tracks and need to work around this limitation. 3. 'odd'/'even' head types not supported for VLBA style tapeforms. For any mode recorded with VLBA style tapeform (14 index positions), the only correct setting of the read and write head types on the 'head.ctl' is 'all'. This will be fixed in a future revision. Please let Ed know if you are missing some tracks and need to work around this limitation. 4. 'chekr' does not check the status of the Mark IV formatter or Mark 5 recorder. Now that most communication problems with the Mark IV formatter have been solved, this will be possible and will be done in the future. 'chekr' support will be implemented for Mark 5 despite communication problems, they will have to be ignored unless they extend beyond a certain amount of time. 5. Extraneous errors when tape is stopped by low tape sensor. When a tape drive has been commanded to move the tape and then stops because it hit the low tape sensor (or when S2 recorders hit the BOT or EOT), 'chekr' will complain periodically that the tape drive is not in the correct state. In principle the FS should be smarter about this. However, if the tape is managed correctly by the schedule this error message should never occur. If it does, then it it an indication that there is either a problem with: (1) the schedule, (2) the check procedures, (3) the recorder, or (4) the tape is too short. If any of these cases apply they should be corrected. It is more likely that this error message will occur when the tape is being controlled manually. In this case, issuing an 'ET' command will convince the FS that the tape drive should be stopped and the error message will cease. 6. 'comm=' command in 'logex' extracts only the first command. The 'comm=' command in 'logex' extracts only the first command commanded and displayed. This problem was noted by Giuseppe Maccaferri (Medicina). 7. S2 error scheme clumsy. The error and status response number reporting scheme for S2 recorders is clumsy. FS errors that have mnemonic 'rl' are mostly error responses from the recorder or the RCL interface library that is used to communicate with the recorder. If the numeric part of an 'rl' error is greater than -130, then it is the error code returned by the recorder. If the numeric part is less than -130, but greater than -300, then add 130 to the value, the absolute value of the result is the error response code from the RCL library. For values less than or equal to -300, a FS error has been detected. Status response codes are all reported with mnemonic 'rz' and the numeric value is the negative of the status response code. In all cases an appropriate error or status message is displayed. These messages are retained in the log. 8. No extra spaces allowed in 'ibad.ctl' file. The format of the 'ibad.ctl' must not contain any leading or embedded spaces. In systems that use the LLP AT-GPIB driver (pre-FS Linux 4), if either the option 'no_untalk/unlisten_after' is misspelled or an incorrect device name is supplied, the driver will cause a segmentation violation when it is initialized and the FS will terminate. Unfortunately there is no way to prevent this problem in a general way; it reflects a limitation in the driver. 9. 'onoff' and 'fivpt' programs hang. The 'onoff' and 'fivpt' programs have been known to 'hang' mysteriously. This seems to be caused by some problem with the 'go' mechanism that is used to restart the program when it pauses to allow a SNAP procedure, such as 'calon' or 'caloff' to execute. The 'go' that is used to restart the program fails for some reason. This has been exceedingly difficult to debug because it is intermittent and fairly rare. There is however a good work around for it. The 'calon' and 'caloff' procedures are called by procedures 'calonfp' and 'calofffp' for 'fivpt' and 'calonnf' and 'caloffnf' for 'onoff'. 'fivpt' and 'onoff' may hang during (or actually just after) the execution of one these procedures that FIVPT and ONOFF will typically hang. If this happens, you will have to terminate the FS to recover. You can prevent it from happening again (for this procedure) by adding the lines: !+1s sy=go fivpt & to the end of 'calonfp' and 'calonfffp'. For 'calonnf' and 'caloffnf', please add: !+1s sy=go onoff & If you see other situations where 'fivpt' and 'onoff' hang, please let Ed know. 10. FS SNAP command pages don't list tape drive suffix numbers. The FS SNAP manual pages and the help pages available through the 'help=' command do not reflect when multiple versions are available with different suffixes depending on the number of drive specified in the control files. For example, there is only a 'tape' page, no 'tape1' or 'tape2' page. However, the help facility will display the version of the command with no suffix when an available command with a suffix is used. For example, if two drives are defined, then 'help=tape1' and 'help=tape2' will work, but 'help=tape' will not and vice-versa if only one drive is defined. 11. LBA rack TPI detector is not usable. The Australian LBA Data Acquisition System currently lacks a functional total power detector though support has been included. To allow approximate system temperature calibration, all the setup commands and the TPI detectors of the modules of a co-existing Mark IV rack are currently also available when the rack type is specified to be LBA4. 12. 'mk5b_mode' and 'bit_streams' commands only report the expected sample rate. The value of the actual clock rate is not read back from the recorder in order to calculate the actual effective sample rate. Consequently, the query log output includes parenthesis around the sample rate as indication that it is not read, but expected. The 'mk5c_mode' command does report the actual sample sample rate. 13. Some 'fmpsee' routines do not generally report file I/O error through the log system for programs within the FS, specifically 'boss', 'incom', and 'aquir'. The 'fmpopen()' routine does use the log system to report errors. Those are the most common errors. However other routines report errors with terminal output. These other routines should eventually use the log system. 14. Some systems calls, particularly in 'mk5cn' and 'dbbcn', use separate UN errors to elaborate on errors in system calls. These should eventually be integrated into the main error message, but whether this makes the errors messages too long (maximum 120 characters) should be considered. A. Reference Appendix for Installation This appendix collects several topics that are useful for installation in general, but are usually not needed for routine updates. A.1. Upgrading from FS versions before the previous stable This sub-section only covers upgrading from "main" branch versions, i.e., versions 9.12.x are excluded. If you are installing FS9 for the first time with this version, please follow the installation instructions in Section 4.5 of the FS9 'Computer Reference' manual. In this case you should also get a copy of the current FS9 'Control Files and Field System Initialization' manual. For reference, the list of the most recent 'critical updates,' since version 9.3.13, is given below. These are updates that must be applied sequentially. Please start with the next update with a later version number than what you have and apply it and the following listed versions before upgrading to the new version. You can find the latest versions of installation notes for these FS versions in the '/usr2/fs/misc' directory. The list of critical updates is: 9.4.0 9.5.3 9.5.12 9.6.9 9.7.7 9.9.2 9.10.4 9.11.6 9.11.8 9.11.19 9.13.2 Strictly speaking you do not need to actually use the source archives (.tgz files) of the previous versions. You can just follow the steps in the upgrade notices for your local files for the corresponding FS versions. However, it can be very helpful to actually install each version to help make sure that all of the upgrade steps have been completed and that the FS will run *and* to test it as described. This can be particularly helpful when the upgrade requires some modifications to your local programs. So it probably best to actually install *and* test each version along the way. This is especially true if you have to upgrade through more than one previous version. Otherwise if a step was overlooked, it might be hard to identify for which version the error was made. You can find the archives for old versions at: http://www.metsahovi.fi/fs/dist/old/ If you have a version older than 9.3.13, please contact Ed (Ed.Himwich@nviinc.com). A.2. Example standard procedure libraries For reference purposes, information about the example station libraries for different equipment configurations is given here. The files are found in /usr2/fs/st.default/proc. They can be referred to and compared to what you have in '/usr2/proc/station.prc'. **ONLY** for new installations (or complete re-installs), you can copy the default version for your equipment to '/usr2/proc' renaming it to 'station.prc' in the process, e.g.: cd /usr2/proc cp -i /usr2/fs/st.default/proc/3station.prc station.prc chmod a+rw station.prc The '-i' option will prompt before overwriting an existing 'station.prc' to give you a chance to recover if you did not realize you already had a 'station.prc' file. The table of correspondence between equipment types and default library names is given next. Equipment Prefix letters Station Library Rack/Drive1/Drive2 k42/k42 k42 k42station.prc k42k3/vlba k42k3v k42k3vstation.prc k42mk4/vlba k42mk4v k42mk4vstation.prc k42mk4/vlbab/vlbab k42mk4vb k42mk4vstation2.prc k42/k5 k42k5 k42k5station.prc lba/s2 ls2 ls2station.prc lba4/s2 l4s2 l4s2station.prc mk3/mk3a 3 3station.prc mk4/mk4 4 4station.prc mk4/mk5a 45 45station.prc mk4/vlba4 4v4 4v4station.prc mk5/mk5b 5 5station.prc none/s2 s2 s2station.prc vlba/s2 vs2 vs2station.prc vlba/vlba v vstation.prc vlba/vlba2 v2 v2station.prc vlba/vlba/vlba v vstation2.prc vlba4/vlba4 v4 v4station.prc vlba4/mk5a v45 v45station.prc vlba4/vlba42 v42 v42station.prc vlba5/mk5b v5 v5station.prc dbbc/mk5b d dstation.prc If an example for your equipment type is listed, please let Ed ('Ed.Himwich@nviinc.com') know so that it can be added. A.3. Cut-and-paste installation tips You can use cut-and-paste to reduce the amount of typing involved in the installation. This reduces the chances of missing required spaces and other easily missed characters (like '.') in the commands. The basic idea is to have two different terminals open, EITHER (preferred) two different xterm terminals either on the local X display or remotely logged in OR (more cumbersome) two different VT text terminals (Control-Alt-F*, where * is 1-6) on the FS computer you are upgrading. You can then switch back and forth between the terminals, reading the instructions as you scroll through them (with 'more' or 'less') on one terminal and entering commands on the other. You can cut-and-paste complicated commands from the terminal with the instructions to the terminal where you are entering commands as needed. You can use 'ssh' or 'su' to 'switch' to users as needed on the terminal where you are entering commands. For example, you can change to being 'prog' by executing: ssh -X prog@localhost or su - prog Please don't forget to log back out when you need to change users again or you may develop a series of 'nested' logins. Any steps that require rebooting will of course completely log out all of your terminals; you will need to re-login again from scratch to continue. At the end of the update, it is recommended that you login as 'oper' on the local X display for the final testing. Please also note that in order to paste into the X display login shell window for 'oper' and 'prog', you typically must use Shift-Insert. If you have any questions about how to cut-and-paste please contact Ed (Ed.Himwich@nviinc.com). A.4. Making a back-up before installing This sub-section has two parts. The first covers back-ups. The second covers using symbolic links to switch between operational and test set-ups. Back-ups Before you begin the upgrade make sure you have a current back-up of your system in case something goes wrong. If you are using one of the FSLx distributions, there are options for each FSL10 (stretch) - See the procedure at: https://nvi-inc.github.io/fsl10/raid.html#_test_upgrade_of_fs_system_updates_or_other_significant_changes FSL9 (wheezy) - If the system is configured as a RAID,please see /usr2/fs/misc/FSL9_RAID.pdf section APPLYING AN UPDATE for directions for applying an update. i FSL8 (lenny), FSL7, (etch), FSL6 (sarge) - If the system is configured as a RAID, please see /usr2/fs/misc/RAID.pdf section APPLYING AN UPDATE for directions for applying an update. FSL5 (woody) - We recommend you use the tar based back-up that is part of the rotating disk back-up scheme. A draft document that describes this method is available in the 'docs' sub-directory on the FS file servers as 'backups2.pdf'. If you have an even older FS Linux distribution, please use the disk-to-disk back scheme described in Section 5.8 of the FS9 'Computer Reference' manual. If you are running one of these FSL distributions and do not have documentation on how to make a back-up, please contact Ed (Ed.Himwich@nviinc.com). If you have SCSI disks, Section 5.7 of the FS9 'Computer Reference' manual has a discussion of drive ID numbers if you are unsure about these. Except for FSL10 (which as a different scheme), you would normally choose to install the update on your primary disk after having made and verified your back-up. Once the installation is complete, has been tested, and used for a little while, you can copy over your back-up with the upgraded primary. If the upgrade fails, you should restore the back-up to the primary for operations. You can then try to upgrade again when it is convenient. In a desperate situation, you can use the back-up for operations. You may choose to install the FS on your back-up disk for testing and then later copy the back-up onto the primary once you are satisfied with the new version. In any event, please be sure to make a fresh back-up (and put it safely away) before attempting an update installation. Using symbolic links *After* you have made a backup (to allow recovery in case something bad should happen), you can use symbolic links to your directories to change between your operational and test directories. This may allow you to more easily switch between an operational and testing configuration. In the following examples, it is assumed that '/usr2/fs-9.13.2' is your operational FS version and the FS you want to test is in '/usr2/fs-test' and that '/usr2/st-1.0.0' is the directory with your station software; you should substitute the correct directories if they are different. All commands must be entered as 'root'. Extra white space has been added only to improve legibility. If you have aliased 'rm' to 'rm -i' and 'mv' to 'mv -i' and 'cp' to 'cp -i', you will prompted to confirm before anything destructive occurs. If so and if everything is set-up properly below, the only cases where you should only be asked to confirm is for deleting the symbolic links in examples iv and v. i. To set-up initially for testing: (make sure the FS is not running) cd /usr2 (make sure there are no existing directories: control-ops, proc-ops, st-1.0.0-ops, control-test, proc-test, st-1.0.0-test, or use different names and adjust this and other examples accordingly) mv control control-ops mv proc proc-ops mv st-1.0.0 st-1.0.0-ops cp -a control-ops control-test cp -a proc-ops proc-test cp -a st-1.0.0-ops st-1.0.0-test ln -sfn control-test control ln -sfn proc-test proc ln -sfn st-1.0.0-test st (then follow the installation instructions, you will be modifying the '-test' versions) ii. To switch temporarily to your operational version (assumed here to use '/usr2/fs-9.13.2'): (make sure the FS is not running) cd /usr2 ln -sfn control-ops control ln -sfn proc-ops proc ln -sfn st-1.0.0-ops st ln -sfn fs-9.13.2 fs (reboot) The above commands (even rebooting if you like) can be put in a script if you need to do this more than once. iii. To switch temporarily to your test version (assumed here to use '/usr2/fs-test'): (make sure the FS is not running) cd /usr2 ln -sfn control-test control ln -sfn proc-test proc ln -sfn st-1.0.0-test st ln -sfn fs-test fs (reboot) The above commands (even rebooting if you like) can be put in a script if you need to do this more than once. iv. When you are satisfied with the testing of the new system (assumed here to use '/usr2/fs-test'), you can switch permanently to it for operations with: (make sure the FS is not running) cd /usr2 rm control rm proc mv control-test control mv proc-test proc mv st-1.0.0-test st-1.0.0 ln -sfn st-1.0.0 st ln -sfn fs-test fs (reboot) Your old operational directories are left for possible future reference. v. To switch permanently to your operational version (assumed here to use '/usr2/fs-9.13.2'): (make sure the FS is not running) cd /usr2 rm control rm proc mv control-ops control mv proc-ops proc mv st-1.0.0-ops st-1.0.0 ln -sfn st-1.0.0 st ln -sfn fs-9.13.2 fs (reboot) Your old test directories are left for possible future reference. A.5. Disk space requirements Please be sure that you have at least 50 MB of free space (use the 'df' UNIX command to check free space) on your '/usr2' partition before starting the upgrade. This would probably only be an issue for stations with 200 MB disks. If you are tight on space, you may want to delete old log files and old versions of the FS (except your most recent one if you can avoid it of course). Since you should have backed-up your system, you can even delete the '*.[oas]' and executable files of your old versions with no risk. You might want to keep the source of the previous versions around for reference if you have room. You can eliminate the non-source files by 'cd'-ing to each of the old FS directories in turn as 'prog' and executing: make rmdoto rmexe as shell command. If you have any questions about how to do this, please contact Ed. A.6. Set operations file permissions It is recommended that your local files for operations (control, proc, log, sched, tle_files directories and their contents) have the default ownership ('oper.rtx') and permissions (for regular files 'rw-rw-r-', for directories 'rwxrwxr-x'). To force this (however, this will not change the "execute/search" bits), please execute the script (as 'root'): /usr2/fs/misc/fix_perm Answer 'y' to the prompt if you wish to proceed. It is a good idea to do this, unless you have purposely changed the ownership and permissions to some other values. If you don't want to restore the defaults, answer 'n' (this is the last chance to abort the execution of the script). If you don't have a '/usr2/tle_files' directory, you will get a message that there is no such directory. A.7. Fix '.prc' file 'define' lines Sometimes due to errors (possibly caused during manual editing, instead of using 'pfmed'), the 'define' statements in '.prc' files can be damaged. This can lead to other problems including causing the contents of procedures being logged every time they are executed rather than just the first time they are used in a given log file. You can use the utility, '/usr2/fs/misc/fix_define', to fix this. You can run it when the FS is *not active* (as 'oper'): cd /usr2/proc /usr2/fs/misc/fix_define -t *.prc in "test" mode to see if there any 'define' statements that need to be fixed. If there are, they will be displayed. If you choose to fix them, you can re-run the second command above without the '-t' flag to apply the fix. An original of each '.prc' file that is changed is retained with an added '.bak' extension.