-------------------------------------------------------------------- How to install PBS using the configure script. 0. Disable SELinux. OpenPBS does not support SELinux. With SELinux enabled, initial start fails with datastore permission error. You can also define proper policy but it is out of scope of this guide. 1. Install the prerequisite packages for building PBS. For CentOS-8 systems you should configure and enable powertools repo for hwloc-devel and libedit-devel packages. You should run the following commands as root: dnf install -y dnf-plugins-core dnf config-manager --set-enabled powertools dnf install -y gcc make rpm-build libtool hwloc-devel \ libX11-devel libXt-devel libedit-devel libical-devel \ ncurses-devel perl postgresql-devel postgresql-contrib python3-devel tcl-devel \ tk-devel swig expat-devel openssl-devel libXext libXft \ autoconf automake gcc-c++ cjson-devel For CentOS-7 systems you should run the following command as root: yum install -y gcc make rpm-build libtool hwloc-devel \ libX11-devel libXt-devel libedit-devel libical-devel \ ncurses-devel perl postgresql-devel postgresql-contrib python3-devel tcl-devel \ tk-devel swig expat-devel openssl-devel libXext libXft \ autoconf automake gcc-c++ For openSUSE systems you should run the following command as root: zypper install gcc make rpm-build libtool hwloc-devel \ libX11-devel libXt-devel libedit-devel libical-devel \ ncurses-devel perl postgresql-devel postgresql-contrib python3-devel tcl-devel \ tk-devel swig libexpat-devel libopenssl-devel libXext-devel \ libXft-devel fontconfig autoconf automake gcc-c++ cJSON-devel For Debian systems you should run the following command as root: apt-get install gcc make libtool libhwloc-dev libx11-dev \ libxt-dev libedit-dev libical-dev ncurses-dev perl \ postgresql-server-dev-all postgresql-contrib python3-dev tcl-dev tk-dev swig \ libexpat-dev libssl-dev libxext-dev libxft-dev autoconf \ automake g++ libcjson-dev For Ubuntu-18.04 systems you should run the following command as root: apt install gcc make libtool libhwloc-dev libx11-dev \ libxt-dev libedit-dev libical-dev ncurses-dev perl \ postgresql-server-dev-all postgresql-contrib python3-dev tcl-dev tk-dev swig \ libexpat-dev libssl-dev libxext-dev libxft-dev autoconf \ automake g++ For macOS systems using MacPorts you should run the following command as root: port install autoconf automake libtool pkgconfig \ expat hwloc libedit libical openssl postgresql14 python310 \ swig-python tcl tk xorg-libX11 xorg-libXt 2. Install the prerequisite packages for running PBS. In addition to the commands below, you should also install a text editor of your choosing (vim, emacs, gedit, etc.). For CentOS systems you should run the following command as root: yum install -y expat libedit postgresql-server postgresql-contrib python3 \ sendmail sudo tcl tk libical chkconfig cjson For openSUSE systems you should run the following command as root: zypper install expat libedit postgresql-server postgresql-contrib python3 \ sendmail sudo tcl tk libical1 libcjson1 For Debian (jessie) systems you should run the following command as root: apt-get install expat libedit2 postgresql python3 postgresql-contrib sendmail-bin \ sudo tcl tk libical1a For Debian (stretch) systems you should run the following command as root: apt-get install expat libedit2 postgresql python3 postgresql-contrib sendmail-bin \ sudo tcl tk libical2 For Debian (buster) systems you should run the following command as root: apt-get install expat libedit2 postgresql python3 postgresql-contrib sendmail-bin \ sudo tcl tk libical3 libcjson1 For Ubuntu-18.04 systems you should run the following command as root: apt install expat libedit2 postgresql python3 postgresql-contrib sendmail-bin \ sudo tcl tk libical3 postgresql-server-dev-all For macOS systems using MacPorts you should run the following command as root: port install expat libedit libical openssl postgresql14-server python310 \ tcl tk 3. Open a terminal as a normal (non-root) user, unpack the PBS tarball, and cd to the package directory. tar -xpvf openpbs-20.0.0.tar.gz cd openpbs-20.0.0 4. Generate the configure script and Makefiles. (See note 1 below) ./autogen.sh 5. Display the available build parameters. ./configure --help 6. Configure the build for your environment. You may utilize the parameters displayed in the previous step. (See note 2 below) For CentOS and Debian systems you should run the following command: ./configure --prefix=/opt/pbs For openSUSE systems (see note 3 below) you should run the following command: ./configure --prefix=/opt/pbs --libexecdir=/opt/pbs/libexec For macOS systems using MacPorts you should run the following commands: export CPATH=/opt/local/include/postgresql14:/opt/local/include export LIBRARY_PATH=/opt/local/lib/postgresql14:/opt/local/lib ./configure --with-swig=/opt/local --with-tcl=/opt/local If PTL needs to be installed along with PBS use the option "--enable-ptl" (see note 5 below) eg ./configure --prefix=/opt/pbs --enable-ptl 7. Build PBS by running "make". (See note 4 below) make 8. Configure sudo to allow your user account to run commands as root. Refer to the online manual pages for sudo, sudoers, and visudo. 9. Install PBS. Use sudo to run the command as root. sudo make install 10. Configure PBS by executing the post-install script. sudo /opt/pbs/libexec/pbs_postinstall 11. Edit /etc/pbs.conf to configure the PBS services that should be started. If you are installing PBS on only one system, you should change the value of PBS_START_MOM from zero to one. If you use vi as your editor, you would run: sudo vi /etc/pbs.conf 12. Some file permissions must be modified to add SUID privilege. sudo chmod 4755 /opt/pbs/sbin/pbs_iff /opt/pbs/sbin/pbs_rcp 13. Start the PBS services. sudo /etc/init.d/pbs start 14. All configured PBS services should now be running. Update your PATH and MANPATH variables by sourcing the appropriate PBS profile or logging out and back in. For Bourne shell (or similar) run the following: . /etc/profile.d/pbs.sh For C shell (or similar) run the following: source /etc/profile.d/pbs.csh 15. You should now be able to run PBS commands to submit and query jobs. Some examples follow. bash$ qstat -B Server Max Tot Que Run Hld Wat Trn Ext Status ---------------- ----- ----- ----- ----- ----- ----- ----- ----- ----------- host1 0 0 0 0 0 0 0 0 Active bash$ pbsnodes -a host1 Mom = host1 ntype = PBS state = free pcpus = 2 resources_available.arch = linux resources_available.host = host1 resources_available.mem = 2049248kb resources_available.ncpus = 2 resources_available.vnode = host1 resources_assigned.accelerator_memory = 0kb resources_assigned.mem = 0kb resources_assigned.naccelerators = 0 resources_assigned.ncpus = 0 resources_assigned.vmem = 0kb resv_enable = True sharing = default_shared license = l bash$ echo "sleep 60" | qsub 0.host1 bash$ qstat -a host1: Req'd Req'd Elap Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time --------------- -------- -------- ---------- ------ --- --- ------ ----- - ----- 0.host1 mike workq STDIN 2122 1 1 -- -- R 00:00 bash$ -------------------------------------------------------------------- NOTES: Note 1: If you modify configure.ac or adjust timestamps on any files that are automatically generated, you will need to regenerate them by re-running autogen.sh. Note 2: It is advisable to create a simple shell script that calls configure with the appropriate options for your environment. This ensures configure will be called with the same arguments during subsequent invocations. If you have already run configure you can regenerate all of the Makefiles by running "./config.status". The first few lines of config.status will reveal the options that were specified when configure was run. If you set envirnment variables such as CFLAGS it is best to do so as an argument to configure (e.g. ./configure CFLAGS="-O0 -g" --prefix=/opt/pbs). This will ensure consistency when config.status regenerates the Makefiles. Note 3: The openSUSE rpm package expands %_libexecdir to /opt/pbs/lib rather than /opt/pbs/libexec which causes problems for the post- install scripts. Providing the --libexecdir value to configure overrides this behavior. Note 4: You need to use a POSIX (or nearly POSIX) make. GNU make works quite well in this regard; BSD make does not. If you are having any sort of build problems, your make should be a prime suspect. Tremendous effort has been expended to provide proper dependency generation and makefiles without relying on any non-POSIX features. The build should work fine with a simple call to make, however, complicating things by using various make flags is not guaranteed to work. Don't be surprised if the first thing that make does is call configure again. Note 5: PTL gets installed in the parent directory of where PBS is installed. For example if you have given install prefix=/opt/pbs, then you can find PTL installation in the directory /opt/ptl . You may need to log out and log in from the terminal for PATH and PYTHONPATH to update. Using valgrind with PBS. ------------------------------------- Here is a set of steps to detect memory errors/leaks within PBS code. 1. Install the valgrind development package. yum install valgrind-devel (zypper for OpenSUSE). 2. Compile Python in a way that valgrind can work with it, as follows: ./configure --prefix= --without-pymalloc --with-pydebug --with-valgrind make; make install 3. Compile PBS with the special python and in debug mode as follows: ./configure --prefix= --with-python= CFLAGS="-g -DPy_DEBUG -DDEBUG -Wall -Werror" 4. Run pbs daemons under valgrind. a) To detect memory errors (not leaks) run pbs daemons as follows: export LD_LIBRARY_PATH=/opt/pbs/pgsql/lib:/opt/pbs/lib:$LD_LIBRARY_PATH valgrind --tool=memcheck --log-file=/tmp/val.out /opt/pbs/sbin/pbs_server.bin b) To detect memory leaks use the supplied leaks suppression file valgrind.supp, as follows: export LD_LIBRARY_PATH=/opt/pbs/pgsql/lib:/opt/pbs/lib:$LD_LIBRARY_PATH valgrind --tool=memcheck --log-file=/tmp/val.out --suppressions=./valgrind.supp --leak-check=full --track-origins=yes /opt/pbs/sbin/pbs_server.bin