Deploying Library Simplifed Services

  1. Home
  2. Deployment Instructions

How to deply Library Simplified applications into your library

Library Simplified application services are designed to be run in the cloud. This provides libraries with no IT infrastructure and few IT management resources the ability to provide high availability services for less.

1: Build your cloud

Goto Amazon and provision server, network and data storage resources .

2: Get the software

Visit the Library Simplified Github Repository here.

3: Configure

Configure the application for your particular envronment and services.

Build your cloud

"Cloud Computing", by definition, refers to the on-demand delivery of IT resources and applications via the Internet with pay-as-you-go pricing. The following page provides step-by-step instructions for deploying any of the Library Simplified server components in the cloud.

Amazon EC2

Amazon EC2 provides a wide selection of instance types optimized to fit different use cases. Instance types comprise varying combinations of CPU, memory, storage, and networking capacity and give you the flexibility to choose the appropriate mix of resources for your applications. Each instance type includes one or more instance sizes, allowing you to scale your resources to the requirements of your target workload. For Library Simplified (at The New York Public Library) we are using a combination of T2 instances, Relational Database Services, Simple Storage Services, and Content Delivery Network services to implement a low cost, low latency, highly scalable Library Simplified environment.

EC2 Server Types
  • T2 instances are burstable performance instances that provide a baseline level of CPU performance with the ability to burst above the baseline. T2 instances are a good choice for workloads that don’t use the full CPU often or consistently, but occasionally need to burst (e.g. web servers, developer environments and small databases). Learn more.
  • Relational Database Service (RDS) is a managed service that makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity, while managing time-consuming database administration tasks, freeing you up to focus on applications and services. RDS automatically receive patches to the database software and backs up to your database, storing the backups for a your user-defined retention period. Learn more.
  • Simple Storage Service (Amazon S3), provides developers and IT teams with secure, durable, highly-scalable object storage. S3 is easy to use, with a simple web services interface to store and retrieve any amount of data from anywhere on the web. Amazon S3 can be used alone or together with other AWS services such as Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Block Store (Amazon EBS), and Amazon Glacier, as well as third party storage repositories and gateways. Learn more
  • Content Delivery Networks (CDNs) are a system of distributed servers that deliver webpages and other Web content to a users based on the geographic locations of the user and the origin of the content delivery server in order to reduce network induced latency. Amazon's CloudFront is a content delivery web service. It integrates with other Amazon Web Services products to give developers and businesses an easy way to distribute content to end users with low latency, high data transfer speeds, and no minimum usage commitments. Learn more.

Environment Diagrams

Frontend

The frontend environment represents the publicly exposed service resources that allow for content syndication and discovery. This is the layer of the application that is accessed by the iOS and Android client applications.

Libary Simlified frontend EC2 environment diagram
Backend

The backend environment represents the service resources that are not exposed publically for data and content management and content hosting and content host integration. This is the layer of the application that is primarily accessed by the circulation server and from time to time by the iOS and Android client applications by way of the OPDS interface.

Libary Simlified backend EC2 environment diagram

Get the Software

Library Simplified Code Repository

The Library Simplified application and source code is available on GitHub. GitHub is a web-based Git repository hosting service, which offers all of the distributed revision control and source code management (SCM) functionality of Git.

Quick Links...
Local environment deployment instructions from repositories

To begin to install the applications you will need to follow the deployment instructions provided in the main repositories wiki at https://github.com/NYPL-Simplified/Simplified/wiki/Deployment-Instructions.

Server software

Server Core

The Server Core contains functionality common between various LS servers, including database models and essential class constants, OPDS parsers, and certain configuration details.
The OA Content Server, Metadata Wrangler, and Circulation Manager all depend on this code base.

Repository: https://github.com/NYPL-Simplified/server_core

.

Circulation Manager

The Circulation Manager is the main connection between a library's collection and Library Simplified's various client-side applications. It handles user authentication, combines licensed works with open access content from the OA Content Server, and data from the Metadata Wrangler.
NOTE: This application depends on the Server Core as a git submodule.

Repository: https://github.com/NYPL-Simplified/circulation.

Optional Server Software

There are optional server components that a library may implement - Open Access (OA) Content Server, Metadata Wrangler and Card Creator. These applications are purely optional because The New York Public Library will be providing functions for some of these servers as a service to implementing libraries so they do not have to install and host the servers. Card Creator is an optional application which will not be provided as a service which needs to be customized on an individual basis by libraries. which will allow libraries to incorporate remote library card application and proof of eligibility documentation in a secure and controlled process so that user data and privacy is secured in transit and at rest during the application review process and promptly deleted from library systems upon acceptance or rejection.

  • OA Content Server at https://github.com/NYPL-Simplified/content_server

    The OA Content Server collects and parses sources and preserves metadata for open access works, serving them up in a feed with verbose OPDS entries.
    NOTE: This application depends on the Server Core as a git submodule.

  • Metadata Wrangler at https://github.com/NYPL-Simplified/metadata_wrangler

    The Metadata Wrangler utilizes and intelligently amalgamates a wide variety of information sources for library ebooks and incorporates them into the reading experience for users by improving selection, search, and recommendations.

    NOTE: This application depends on the Server Core as a git submodule.

  • Card Creator at https://github.com/NYPL-Simplified/card-creator

    The Card Creator application allows libraries to incorporate remote library card application and proof of eligibility documentation in a secure and controlled process so that user data and privacy is secured in transit and at rest during the application review process and promptly deleted from library systems upon acceptance or rejection.

Client Application Software

The client applications are currently contained in open repositories accept for the those DRM compponents that are derivative products which are in private repositories. Partner libraries may obtain access to the private repositories and the closed repositories for DRM that require a commercial license. Once the commercial license is obtained users can gain access to the code libraries that link to the commercial binaries distributed independently by the commercial DRM vendor.

Mobile Clients
Web Clients

NOTE: The application can be built without DRM support as an Open Access (non DRM) content e-reader. However, a Readium License is still required. This license can be obtained directly from the Readium Foundation at special Non-Profit rate. Additionally Minitex provide a special Non-profit license as well specifically to public, state and academic libraries as well at a reduce price. Visit their product page here to learn more.

IT Orchestration and Containerization

To make initital implementation and maintenance easier and in keeping with modern DeveOps practices for technology products the Libary Simplified community practices Continuous Integrations and Continuos Delivery of the midleware through the use of operating system virtualization using containerizationand community managed open source IT Orchestration tools such as Ansible for automating software provisionsing, configuration and application deployment.

Docker

Containers

Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation of operating-system-level virtualization on Linux. Docker can package an application and its dependencies in a virtual container that can run on any Linux server. This helps enable flexibility and portability on where the application can run, on premises, in a public cloud, on a private cloud, or bare metal machines.

Ansible Logo

Ansible Playbooks

The Ansible Playbooks are the files that are used by the open source automation engine, Ansible. The playbooks provide the instructions and linformation necessay to remotely provision the necessary Amazon Web Services environment to host the application. They also automate the deployment of the Library Simplified Circulation Manager Docker Containers, application servers, databases and other related applciations.

Using a deployment management tool like Jenkins, a docker containter can be pulled down by the repsective library from Docker Hub as a self contained application container onto your infrastructure (for example EC2.) This simplifies application implementation and deployment by prepackaging the application and all the required stack technolgies and dependencies with out having to manualy configure or run PIP and GIT commands. Simply pull the latest compatible container version, update the JSON configuration file during the deploy process. For certain versions you may have to run some update scripts on your DB server.

Configure

The Configuration instructions can be found on github on the main project's wiki at https://github.com/NYPL-Simplified/Simplified/wiki/Configuration.

Additionally, guides to configureing youer library, different conetnet and meta dat integrations can be found at www.librarysimplified.org/documents.

Maintaining the servers

The /bin directory contains autmated scripts that should be run every 5 minutes to keep your collections updated as well as fix data from time to time. The wiki link below lists the scripts and their purpose. Unless you are running a separate instance of Metadata Wrangler and OA Content Server as opposed using the NYPL public service, those instructions and scripts can be ignored. They will be done as part of NYPL's maintenance.

Automated Jobs
https://github.com/NYPL-Simplified/Simplified/wiki/AutomatedJobs#jobs-that-should-be-run-as-needed

Management and Monitoring Tools

The ability to monitor your cloud environment and application services is important. Libraries and their technical staff need simple tools for keeping your ebook service running and scallable so that users have a wonderful and consistent Library Ebook service experience. While you may have your own tools that you prefer to manage your enterprise or cloud systems, the following are examples of what the Library Simplifed Team use for the project, Open eBooks and the New York Public Library.

Environment Management
Product Delivery Purpose
Enterprise This tool allows you to manage your cloud infrastructure and application service configurations.
Automated Build, Test and Deployment for Continuous Integration and Deployment
Product Delivery Purpose
Enterprise This tool is used to manage continuous deployment of the application between your development environment (if needed) QA/pre-production environment and production service environment.
SaaS This service provides a tool to help manage continuous integration and testing of your pre-compiled mobile applications (iOS, and Android) as well as other test automation processes.
IaaS This service is used to provide mobile device environments and test infrasturucture for conducting automated testing. This service can provide iOS and Android test device instances for a mulititude of device manaufactures and OS combinations.
Enterprise Maven leverages the concept of a repository by retrieving the artifacts necessary to build an application and deploying the result of the build process into a repository. Maven uses the concept of structured repositories so components can be retrieved to support the build. These components or dependencies include libraries, frameworks, containers, etc.
IaaS The Central Repository is the default repository for Apache Maven, SBT and other build systems and can be easily used from Apache Ant/Ivy, Gradle and many other tools.
Enterprise Nexus manages software "artifacts" required for development. If you develop software, your builds can download dependencies from Nexus and can publish artifacts to Nexus creating a new way to share artifacts within an organization.
Application Monitoring and Logging
Product Delivery Purpose
SaaS This tool provides insights into your application performance through log file mining and visualization. You can create queries and alerts based on queries to detect logged events or sequences of events for reporting and monitoring.
SaaS This tool is FREE for non-profits and a must have for logging stack trace level issues. This tool can be used across the entire application stack on the backend servers applications and mobile clients.
SaaS This tool is used to monitor the server application's performance. This tool can help you identify process bottlenecks in your application.
SaaS This tool provides application analytics in terms of user insights. This tool is used in the clients to anonymously measure click through of the applications so as to improve user experience.
```