PGXN Project Status

Track the progress of PGXN development! This is the PGXN project plan. We'll keep it up-to-date as development progresses, and as tasks evolve and change.

I. Master Mirror

• Set up server
• Set up rsync-able distributable directory

The goal of this phase is to simply get the master mirror working and mirrors replicating from it. The structure of what’s to be mirrored will not yet have been worked out, but this will be the time to integrate with the PostgreSQL mirror system as well. 5% of project.

II. Distribution

• Design directory structure
• Set up distribution to master mirror
• Set up mirror indexing
• Set up cron job to validate mirrors

This phase will see the design of the directory structure, including how things will be related (via symlinks perhaps?). It will be the job of the software created in this phase to deliver files to the master mirror server and to check the health of existing mirrors. 8% of project.

III. Extension Database

• Design database schema
• Design database API
• Automate database/index dump

The root index for PGXN will be a PostgreSQL database. It will be dumped to a file regularly and included in the mirrors. This will allow clients to be able to build a database with the complete metadata contents. Other metadata formats (JSON, CSV) may also be generated.

The database will also include an API that the upload server will use to udpate the index. This API will not be included in the dump, but included in a private schema. 12% of project.

IV. Upload Server

• Design UI
• Implement Upload Webapp
  • Implement authentication (using basic auth)
  • Implement distribution cleansing into .pgz (zip) package
  • Drop .pgz into proper directory
  • Update database records
  • Allow permissions editing
  • Provide a comprehensive Web API
• Deploy to manager.pgxn.org
• Test, test, test, test

Completion of this phase will make PGXN a real option for PostgreSQL extension distribution. When this phase is completed, registered users will be able to login to the upload server, upload a distribution tarball, and allow other users to be listed as co-release managers. Uploaded files will be indexed by the upload server, inserting the proper values into the extension database, and then delivered to the root directory for distribution to the master mirror server.

The upload server will also provide a complete Web API to allow third-party applications to interrogate the metadata database, as well as authenticate a user and release a tarball. The command-line client developed in phase VI will be the first utility to use this API. 15% of project.

V. Search Site

• Set up cron job to sync from the master server and build the index database
• Design UI
• Implement Webapp
• Features:
  • Full text search of all extensions
  • Pages for each extension upload
  • Pages for each documentation file included in the tarball
  • Pages for each extension release manager
• Write Unit/Integration/Acceptance tests
• Set up deployment strategy
• Launch

Modeled on The CPAN Search site, the search site will be the primary online presence for PGXN. If one needs a PostgreSQL extension, it should be the first stop in the search for existing extensions. Furthermore, each extension will have an intuitive, descriptive URL, as will every page of extension documentation, for each version of an extension. This will make it easy to create links to various documentation pages from other sites, thus boosting Google search rankings for extensions.

Once this phase is completed, it will be time to publicly launch the PGXN service, as any changes required of the infrastructure from the previous phases will be hammered out and finalized. This is because the search site will effectively be a client of PGXN, and as the first client, will likely require changes from the PGXN’s API and database structure. But once search is done, the PGXN API and database can be blessed as 1.0 and the whole project launched as 1.0. 30% of project.

VI. Command-Line Client

• Design the interface
  • Supported commands
  • Command-line options
• Implement the client
  • Set up configuration options and storage
  • Allow metadata interrogation to be pluggable
    • Go with Web API to start
    • Maybe Download PostgreSQL dump file for local index database
  • Use PGXS to build downloaded extensions
    • Initially, that means make && make install && make installcheck
    • Allow flexibility to support other build systems in the future

The final part of the project will be the creation of a command-line client modeled on cpanminus and CPAN.pm. It will do its best to be self-configuring, so that the user does not have to do anything to get started with it. But it will be highly configurable, so that users can select mirrors to use, different installed versions of PostgreSQL, etc. 30% of project.