checklist for doing a release:

1. Check weewx.conf for local changes.  In particular, check to make sure:
     1. debug=0 in weewx.conf
2. Make sure the version is correct
     1. modify pyproject.toml
     2. make version
3. Make sure all changes have been logged
     1. doc_src/changes.md
     2. doc_src/upgrade.md
4. Build the documentation
     1. make build-docs
5. Create the packages
     1. make src-tarball
     2. make pypi-package
     3a. make debian-changelog
     3b. make debian-package-via-vagrant GPG_KEYID=xxx
     3c. make apt-repo-via-vagrant GPG_KEYID=xxx
     4a. make redhat-changelog
     4b. make redhat-package-via-vagrant GPG_KEYID=xxx
     4c. make yum-repo-via-vagrant GPG_KEYID=xxx
     5a. make suse-changelog
     5b. make suse-package-via-vagrant GPG_KEYID=xxx
     5c. make suse-repo-via-vagrant GPG_KEYID=xxx
6. Check each installation method
     1. do a git install + upgrade
     2. do a pip install + upgrade
     3. do a debian install + upgrade
     4. do a redhat install + upgrade
7. Commit and tag
     1. git commit -m "Version X.Y.Z"
     2. git push
     3. git tag vX.Y.Z
     4. git push --tags
     5. create a release at github.com
8. Upload wheel to pypi.org
     1. make release-pypi
9. Upload to weewx.com
     1. make upload-docs
     2. make stage-all
10. Move files on the server from staging to production
     1. make release-all
11. If the minor version changes, update doc links in the website repository
     1. change URL= in docs/*.html
     2. change version in docs/weedocs.js
     3. change symlink docs/latest
12. Announce the release to the weewx user's group.


pre-requisites ----------------------------------------------------------------

Building the documentation requires mkdocs, which is installed using pip.
Signing packages requires gpg and local copy of private+public keys.
Verifying the packages requires rpmlint/lintian.
The debian repo management requires aptly.
The redhat repo management requires createrepo.
The suse repo management requires createrepo.

To build the docs:
  pip3 install --user mkdocs
  pip3 install --user mkdocs-material

To build pypi package install the following (see the pypi section):
  pip3 install --user poetry

To build debian package install the following (use 'apt install'):
  git
  rsync
  gpg
  debhelper
  lintian
  aptly           only needed if you will create and update repositories

To build redhat package install the following (use 'yum install'):
  git
  rsync
  gnupg
  rpm-build
  rpm-sign
  rpmlint
  rpmdevtools
  createrepo_c    only needed if you will create and update repositories

To build suse package install the following (use 'zypper install'):
  git
  rsync
  rpm-build
  rpmlint
  createrepo_c    only needed if you will create and update repositories


howto -------------------------------------------------------------------------

how to update the version number:
  Change the version number in pyproject.toml
  make version                  # this propagates the version number everywhere
  git commit -a -m "bump to version x.y.z"
  git tag vx.y.z
  git push --tags

how to build wheel and source tarball:
  make pypi-package

how to build debian package:
  make debian-changelog
  emacs pkg/debian/changelog    # add package-specific changes, if any
  git commit -m "update deb changelog" pkg/debian/changelog
  make debian-package

how to build redhat package:
  make redhat-changelog
  emacs pkg/changelog.el       # add package-specific changes, if any
  git commit -m "update redhat changelog" pkg/changelog.el
  make redhat-package

how to build redhat package:
  make suse-changelog
  emacs pkg/changelog.suse     # add any package-specific changes, if any
  git commit -m "update suse changelog" pkg/changelog.suse
  make suse-package

how to build redhat packages with custom rpm revision:
  make redhat-changelog RPMREVISION=2
  make redhat-package RPMREVISION=2
  make pull-yum-repo
  make update-yum-repo RPMREVISION=2
  make push-yum-repo
  make release-yum-repo
suse also uses RPMREVISION, but debian uses DEBREVISION.  this is useful when
there is a change to asset(s) in the packaging, but not part of weewx itself.

how to install using pip:
  make pypi-package
  pip install dist/weewx-x.y.z-py3-none-any.whl
  weectl station create

how to install/remove debian:
  apt-get install weewx         # install with apt
  apt-get remove weewx          # remove with apt
  apt-get purge weewx           # purge removes /etc/weewx and debconf settings
  dpkg -i weewx_x.y.z-r.deb     # install
  (apt-get install weewx)       # finish install if dependencies failed
  dpkg -r weewx                 # remove
  dpkg -P weewx                 # purge

how to install/remove redhat:
  yum install weewx-x.y.z-r.rpm [--nogpgcheck]    # install with yum
  yum remove weewx                                # remove with yum
  rpm -i weewx-x.y.z-r.rpm                        # install with rpm directly
  rpm -e weewx                                    # remove with rpm

how to install/remove suse:
  zypper install weewx-x.y.z-r.rpm [--nogpgcheck] # install with zypper
  zypper remove weewx                             # remove with zypper
  rpm -i weewx-x.y.z-r.rpm                        # install with rpm directly
  rpm -e weewx                                    # remove with rpm

to display debconf variables:
  sudo debconf-show weewx

to manually purge debconf variables:
  echo PURGE | sudo debconf-communicate weewx

to sign rpm packages you need .rpmmacros in your home directory:
~/.rpmmacros
  %_gpg_name  YOUR_NAME_HERE

to sign the apt Release using key 'XXX':
  gpg -abs -u XXX -o Release.gpg Release

to sign the RPM repository metadata using key 'XXX':
  gpg -abs -u XXX -o repomd.xml.asc repomd.xml

to generate gpg key used for signing packages:
  gpg --gen-key
  gpg --list-keys
  gpg --list-secret-keys

to export the text version of a public key:
  gpg --export --armor > username.gpg.key

list keys known by rpm:
  rpm -q --gpg-pubkey
  rpm -q --gpg-pubkey --qf '%{NAME}-%{VERSION}-%{RELEASE}\t%{SUMMARY}\n'

delete keys known by rpm:
  rpm -e gpg-pubkey-XXXXX


build environment -----------------------------------------------------------

The build for each package must be done in the appropriate operating system,
and we do not support cross-platform building.  For example, to build the
debian package you must build on a debian system.  There are vagrant files
in the weewx source tree to automate the setup of build environments for
creating weewx packages.


howto: build the documents --------------------------------------------------

Prerequisites:
    - Python 3.7 or greater, with pip installed
    - Install mkdocs:
        python3 -m pip install mkdocs
    - Install the theme "material" for mkdocs:
        python3 -m pip install mkdocs-material

Steps:
    - make build-docs


howto: build and publish wheels to pypi.org -----------------------------------

Prerequisites:
    - Python 3.7 or greater, with pip installed
    - Install poetry:
        curl -sSL https://install.python-poetry.org | python3 -
    - Get an API token from pypi.org
        See https://pypi.org/manage/account/token/
    - Tell poetry to use it:
        poetry config pypi-token.pypi pypi-substitute-your-pypi-key

Steps:
    - Build the wheel
        make pypi-package
    - Publish to pypi.org
        make upload-pypi


howto: update package repositories --------------------------------------------

There are two repositories for each platform, one for production and one for
testing.  For example, for debian we have apt and apt-test.  When preparing
a debian, redhat, or suse package for release, first do testing by installing
from a local deb/rpm.  When you are ready to release to a repo, the steps are:

1) make pull-xxx-repo - this will copy the production repo to local disk

2) make update-xxx-repo - this will copy staged deb/rpm to local repo, then
   update repo info and sign with gpg keys

3) make push-xxx-repo - this will copy the local repo to remote test repo

4) make release-xxx-repo - replace production repo with test repo

If you are doing alpha or beta releases, you can push those to the remote test
repo, but do *not* release them.  After doing alpha and/or beta releases, be
sure to pull the production repo before you get out of alph/beta, then push to
the test repo.  Otherwise you risk pushing alpha/beta releases into the
production repo, which will break many systems.

We must keep alpha/beta releases out of the production repo, since they are
not handled properly by some package management software.  For example, yum
thinks that 5.1.0-b6-1 is newer than 5.1.0.


apt repository details --------------------------------------------------------

The weewx DEBs are tightly coupled to the major python version (2 or 3), and
the availability of pre-built python modules in the operating system release.
This means that the weewx DEBs span not only minor operating system updates,
but also major operating system releases.

aptly has two different mechanisms for doing a 'publish': switch or update.
we use snapshots, and publish using 'publish switch', rather than publishing
using a simple 'publish update'.

There are two apt repositories: python2 and python3

to do apt repo updates you must first install aptly:
  https://www.aptly.info/download/
for example, on debian:
  echo "deb http://repo.aptly.info/ squeeze main" | sudo tee /etc/apt/sources.list.d/aptly.list
  wget -qO - https://www.aptly.info/pubkey.txt | sudo apt-key add -
  sudo apt-get update
  sudo apt-get install aptly

create local debian repo using aptly:
  aptly repo create -distribution=squeeze -component=main -architectures=all python2-weewx
  aptly repo create -distribution=buster -component=main -architectures=all python3-weewx

put a bunch of deb files into an empty apt repo:
  for f in `ls distdir`; do aptly repo add python2-weewx distdir/$f; done

create a snapshot:
  aptly snapshot create python-weewx-x.y.z-n from repo python2-weewx
  aptly snapshot create python3-weewx-x.y.z-n from repo python3-weewx

publish using snapshot:
  aptly publish -architectures=all snapshot python-weewx-x.y.z-n python2
  aptly publish -architectures=all snapshot python3-weewx-x.y.z-n python3

update using 'publish switch':
  aptly repo add python2-weewx dist/python-weewx_x.y.z-n_all.deb
  aptly snapshot create python-weewx-x.y.z-n from repo python2-weewx
  aptly publish switch squeeze python2 python-weewx-x.y.z-n

  aptly repo add python3-weewx dist/python3-weewx_x.y.z-n_all.deb
  aptly snapshot create python3-weewx-x.y.z-n from repo python3-weewx
  aptly publish switch buster python3 python3-weewx-x.y.z-n

update using 'publish update':
  aptly publish repo -architectures=all python2-weewx squeeze
  aptly repo add python2-weewx dist/squeeze/python-weewx_x.y.z-n_all.deb
  aptly publish update squeeze python2

  aptly publish repo -architectures=all python3-weewx buster
  aptly repo add python3-weewx dist/buster/python3-weewx_x.y.z-n_all.deb
  aptly publish update buster python3

clone the published apt repo to local space:
  mkdir -p ~/.aptly
  rsync -Oarvz --delete USER@weewx.com:/var/www/html/aptly-test/ ~/.aptly

synchronize local aptly changes with the published apt repo:
  rsync -Oarvz --delete ~/.aptly/ USER@weewx.com:/var/www/html/aptly-test

switch from testing to production (this is done at weewx.com):
  rsync -Oarvz --delete /var/www/html/aptly-test/ /var/www/html/aptly

for clients to use an apt repo at weewx.com:
  curl -s http://weewx.com/keys.html | sudo apt-key add -
  echo "deb [arch=all] http://weewx.com/apt/ squeeze main" | sudo tee /etc/apt/sources.list.d/weewx.list
  echo "deb [arch=all] http://weewx.com/apt/ buster main" | sudo tee /etc/apt/sources.list.d/python3-weewx.list


yum/zypper repository details -------------------------------------------------

The redhat (yum/dnf) and suse (zypper) repositories have similar structure and
behavior, but the RPMs are not interchangeable.  In each case, the weewx RPMs
are tightly coupled to the major operating system release.  The weewx RPMs do
*not* track minor or bugfix releases of the operating system.

create yum repo:
  mkdir -p ~/.yum/weewx/{el7,el8,el9}/RPMS

update local yum repo with latest rpm:
  cp *.el7.rpm ~/.yum/weewx/el7/RPMS
  createrepo -o ~/.yum/weewx/el7 ~/.yum/weewx/el7
  cp *.el8.rpm ~/yum/weewx/el8/RPMS
  createrepo -o ~/.yum/weewx/el8 ~/.yum/weewx/el8
  cp *.el9.rpm ~/yum/weewx/el9/RPMS
  createrepo -o ~/.yum/weewx/el9 ~/.yum/weewx/el9

clone the published yum repo to local space:
  mkdir -p ~/.yum
  rsync -Oarvz --delete USER@weewx.com:/var/www/html/yum-test/ ~/.yum

synchronize local yum changes with published yum repo:
  rsync -Oarvz --delete ~/.yum/ USER@weewx.com:/var/www/html/yum-test

switch from testing to production (this is done at weewx.com):
  rsync -Oarvz --delete /var/www/html/yum-test/ /var/www/html/yum


notes -------------------------------------------------------------------------

there are multiple changelogs:
  docs/changes.md - definitive changelog for the application
  pkg/debian/changelog - changes to the debian packaging
  pkg/changelog.el - changes to the redhat packaging
  pkg/changelog.suse - changes to the suse packaging

the debian changelog *must* have a version number that matches the app version.
the redhat package will build if the version numbers do not match.  use the
redhat-changelog and debian-changelog targets to ensure that changelog versions
match the application version for a release.

there are many ways to build a debian package.  first tried dpkg (uses DEBIAN
dir and is fairly low-level) but that does not create changes and source diffs.
then dried dpkg-buildpackage (uses debian dir and is higher level) but misses
the config and templates.  ended up using dpkg-buildpackage with some manual
(scripted) file manipulation.

prefer to use expressive but no overly verbose output in scriptlets and
maintainer scripts.  many redhat scriptlets seem to be silent, perhaps only
emitting messages when there is a failure.  unfortunately, most of our failures
have been due to things that we never even predicted, let alone created error
traps for.  so having output from each step helps us figure out where a
failure came from.  apparently redhat will add better logging/capture from
scriptlets in redhat 10 - the anaconda installer already does a pretty good
job of capturing and isolating output from scriptlets, but redhat plans to add
similar functionality to dnf in redhat 10.


shebang -----------------------------------------------------------------------

both debian and redhat deprecate the use of '/usr/bin/env python' as the
shebang - they want a very tight binding to the operating system python as the
shebang.  in fact, since late 2019 redhat build tools see any shebang other
than a tight coupling to the operating system's python as an error and refuse
to accept it.  however, the bsd platforms prefer this as the shebang, and the
other approach fails on bsd.  macos uses something else.

since weewx5 supports any python 3.6+, older weewx supports python 2.7 and any
python 3.5+, weewx does not have to be tightly coupled to any specific python
installation on the system.

so the source code should use the '/usr/bin/env python' shebang.  on platforms
that refuse to accept this, the package builder will replace this with
whatever that platform will accept.  for pip installs, pip does the shebang
mangling and sets entry points that are appropriate for its configuration.
for everything else, using the env in shebang and making the entry points
executable enables either 'python foo.py' or 'foo.py' invocation.


defaults ----------------------------------------------------------------------

the /etc/default/weewx plus shell stubs in /usr/bin/wee* is used in deb/rpm
installations to provide python flexibility, so that users can use a single
weewx installation to experiment with different python versions. this is
particularly helpful when running weewx directly from source in a git clone,
it also works in the deb/rpm installs where the python is managed separately
from the system's python, as well as the non-linux, non-pip installations.

unfortunately, this can cause problems during an installation.  if the pre/post
install scripts invoke /usr/bin/weectl instead of the python code directly,
they can end up getting python2 or a python3 that does not have the right
modules installed.  so maintainer scripts and scriptlets must ensure that they
use a known-working python.


signing packages --------------------------------------------------------------

gpg is used to sign the deb repository and rpm packages.

When signing RPMs, gpg info must match the name and email in the latest package
changelog entry.

When signing apt Release using aptly, beware that aptly uses the first gpg key
that it finds.  that might not be what you want.

SHA1 is no longer acceptable for signing, so be sure that your gpg keys and
the signing command use SHA256 instead.  This should be the default when
building on redhat9 and later.  If it is not the default, then it can be forced
with a change to the signing macro in .rpmmacros - add the line
--digest-algo sha256

%__gpg_sign_cmd			%{__gpg} \
	gpg --no-verbose --no-armor \
	%{?_gpg_digest_algo:--digest-algo %{_gpg_digest_algo}} \
	--no-secmem-warning \
	--digest-algo sha256 \
	%{?_gpg_sign_cmd_extra_args:%{_gpg_sign_cmd_extra_args}} \
	-u "%{_gpg_name}" -sbo %{__signature_filename} %{__plaintext_filename}

In the debian world, you sign the repository (specifically the file 'Release'),
not the individual .deb files.  So if you need to re-sign, you re-build and
re-sign the repository; there is no need touch the individual .deb files.
Signing is controlled by the -us and -uc options to dpkg-build.  If you do not
specify those options, then dpkg-build will try to sign the .dsc, .buildinfo,
and .changes files.  The .deb itself is not signed.

SUSE wants you to sign the RPMs as well as the repository metadata.  The meta
data are in repomd.xml, and a fully signed repository must include the files
repomd.xml.asc and repomd.xml.key.  So although it is possible to use one key
for the meta data and different keys for the RPMs, it is probably best to sign
with a single, shared key.

On SUSE, zypper keeps the repo information in a local cache /var/cache/zypp/raw


init --------------------------------------------------------------------------

For the packaged installers, the installer must "do the right thing" with
respect to starting, restarting, and enabling the weewxd daemon.  This does
not necessarily align with platform policies.

Fedora docs say that a service may be enabled if it does not alter other
services, does not fail, does not require configuration before starting, and
does not listen for outside connections.  However, some systemd docs say that
you should use presets instead of enable/disable, so that 'spin' maintainers
can control the app using presets.  So depending how you read this, you could
argue that weewx could be enabled but not run upon install.

Debian just enables and runs everything upon install.

The weewx packages *always* enable and *always* run weewx on a new install.
On an upgrade, the suse/redhat installer will run weewx if weewx was running
before, while the debian installer will always run weewx.  This is because we
can detect whether weewx was running on suse/redhat, but we cannot on debian.

For a new install, we only enable and start the primary unit.  For an upgrade,
on a system with systemd we remove any sysv bits and migrate them to systemd.

On systems with systemd (redhat, suse, debian):
- new install
  - start weewxd
  - enable weewxd
- upgrade from 4.x
  - remove any SysV scripts - weewxd and weewx-multi
  - start weewxd (suse/redhat: only if weewxd was already running)
  - enable weewxd
- upgrade from 5.x
  - start weewxd (suse/redhat: only if weewxd was already running)
  - enable weewxd

On systems with SysV (debian):
- new install
  - start weewxd
  - enable weewxd
- upgrade from 4.x
  - replace weewx and weewx-multi with weewx
  - start weewxd
  - enable weewxd
- upgrade from 5.x
  - start weewxd
  - enable weewxd


unit tests --------------------------------------------------------------------

prerequisites:

  - python 3.7
  - python-usb
  - pyephem

to set up mysql server with user and permissions for testing:

  make test-setup

to run all unit tests:

  make test
    (note: do not run this as root)

to clean up after running tests:

  make test-clean

guidelines:

unit tests should put transient files in /var/tmp/weewx_test


testing -----------------------------------------------------------------------

what to test when creating debian and redhat packages:
  install, upgrade, remove, purge
  install, modify files, remove
  install previous release, modify files, upgrade, remove

Using pip:

- new install to user space
  make pypi-package
  pip install dist/weewx-x.y.z-py3-none-any.whl --user
  weectl station create

- upgrade user data
  modify ~/weewx-data/weewx.conf
  weectl station upgrade

- new install using pip to /opt/weewx
  make pypi-package
  sudo pip install dist/weewx-x.y.z-py3-none-any.whl
  sudo weectl station create /opt/weewx/weewx.conf

- upgrade using setup.py to /opt/weewx
  setup.py install home=/opt/weewx
  modify /opt/weewx/weewx.conf
  setup.py install home=/opt/weewx

on centos and suse:

- new install using rpm
  rpm -i weewx_x.y.z.rpm

- upgrade using rpm
  rpm -i weewx_r.s.t.rpm
  rpm -U weewx_x.y.z.rpm

- upgrade using rpm with extensions installed
  rpm -i weewx_r.s.t.rpm
  wee_extension --install cmon
  rpm -U weewx_x.y.z.rpm

debian:

- new install usinb dpkg
  dpkg -i weewx_x.y.z.deb

- upgrade using dpkg take maintainer's version of weewx.conf
  dpkg -i weewx_r.s.t.deb
  modify /etc/weewx/weewx.conf
  dpkg -i weewx_x.y.z.deb

- upgrade using dpkg use old version of weewx.conf
  dpkg -i weewx_r.s.t.deb
  modify /etc/weewx/weewx.conf
  dpkg -i weewx_x.y.z.deb

- reconfigure using dpkg
  dpkg-reconfigure weewx

all platforms:

- installation and removal of extensions
  weectl extension install https://github.com/matthewwall/weewx-cmon/archive/master.zip
  weectl extension install ~/weewx-data/examples/pmon
  weectl extension uninstall cmon
  weectl extension uninstall pmon

- reconfigure
  weectl station reconfigure
  weectl station reconfigure --driver=weewx.drivers.vantage --no-prompt