It has been quite a while since the last version of SADE – the TextGrid flavoured version of the »Scalable Architecture for Digital Editions« – was published. Here is a new version, with a new deployment strategy and a complete new architecture. Everything is new in 2018.

Once this version of SADE was derived from a development that Alexander Czmiel (TELOTA at BBAW) promoted in »Editio ex machina - Digital Scholarly Editions out of the Box«. There was a joint development together with ÖAW and SUB Göttingen. According to a bullet point here SADE was used in circa 30 projects. With an additional package the software was prepared to become the web-publishing framework of TextGrid, a larger infrastructure to prepare editions in the digital age.

As I started working with the package, it was not compatible with the standard application environment of an eXist-db. E.g. the templating engine was forked from an early version and customized, features (I have never used) require a modification of the build process and and and.

For some time i struggled with these things until i realized that the time for major changes was over for the concrete project I was working on. The nearly complete rewrite i present in this post was finished to late. Anyway, the new version contains the know-how we collected over the years at SUB especially working on the TextGrid project. In order to prepare a cleaner code i gave some modules the chop. The result is a package that complies to the standards established in the eXist-db environment. It is easy to build and deploy and with a new module that ships the assets, it needs less resources even if there are hundreds of projects deployed in a single database instance.

Not everything became a victim of the wrecking ball. There are new features like the support of multi-language websites and a Confluence Wiki parser as well. But first things first.

Concepts

The idea was to share a core of methods between several projects and host them on the same server, within the same container and the same database. An architecture like this might save you some time (= money) on the maintenance side of software life, but it assumes that there is no jokester to check in insecure code in this environment. One further assumption might be that any customization can be done via static scripts that ships with the data, namely XSLT.

We had some troubles implementing new tools – mostly improvements to the frontend. Here is one example: For the SemToNotes integration we had to interfere with the mulitviewer – a central component with the task to route sources to their corresponding modules or functions. We may did a mistake here by inserting this module as a part of the SADE application instead of preparing a complete new package, but on the other side we were not ready to publish complete packages and all of their interdependencies in our ecosystem. Creating a module within the package was a sufficient way to reduce the overhead here. May be this was a wrong decision. But in the end we have seen that there is no comfortable way to integrate a new component.

Dealing with TEI files the default behavior of the multiviewer was to call the standard TEI XSLT and provide the output to the controller (MVC like). That’s it. Regarding to the namespace of the file, the multiviewer selected different XSLT stylesheets, but when it comes to a different treatment of nodes within a single namespace, we never evaluated against a XPath provided by the project, we always wrote conditionals in the multiviewer’s code. So we cut the branch we were sitting on.

In two projects we also altered the publication process (provided by a standalone package) and added a few functions to the data transfer process from the TextGrid Repository to the SADE instance. Besides the SADE code package, we changed the code of this second package and there was no convenient way to share updates.

Solutions

In order to prepare a flexible tool that fits the requirements of the future projects I decided to throw out as much as possible. This also gives us the chance to rewrite some code, provide a documentation (no, not a better documentation, but at least one!), to include tests and custom error codes the application can display in case of a typical mistake.

The package that provides an endpoint for TextGrid’s SADE Publisher tool (textgrid-connect) is now part of the SADE package. The frontend in the TextGrid Laboratory now comes with the look and feel of the website you like to view the data in. It is more easy to manipulate the publishing process like we did in two projects.

Improvements

I take advantage of the error() functions to send customized (and hopefully helpful) messages in case of missing parameters and all the typical mistakes and errors I can foresee. With custom error codes developer can find the danger spot in the source quickly. All messages from the database are now passed to the Lab frontend.

We updated the template from an very old (but also responsive version) to the newest available with full backwards compatibility.

The complete documentation is shipped with the application.

A new feature allows to fork and download a project via the Lab frontend. So it is easy to start developing new features or working on bugs. By overriding a git fork, the development stays in touch with the upstream.

A parser for Dokuwiki and Confluecne Wiki is now available to all projects. This allows the maintenance of static texts via comfortable interfaces. Changes are tracked via feed and the website is updated immediately.

The website is now prepared for multi-languages. You can add translations in an i18n style and all html:meta and GET parameters will be set automagically.

Preconfigured indexes are available for both: TextGrid metadata and TEI data. The charmap and synonyms are configurable via text files (and can be maintained within the TextGridLab).

Sources and Packages

Via GitLab we provide the source code and all packages (releases and snapshots).

https://gitlab.gwdg.de/SADE

Besides the deprecated packages there are:

• SADE
• build
• assets
• Dialogo-Medieval (= sample customization)

SADE

The main repository contains all the code and documentation. All bugs and feature requests are tracked here. If you are familiar to eXist-db or you don’t hesitate to setup your own, you can use the artifacts from this repo, but you have to take a look on the dependencies.

assets

Here we maintain all static stuff like images, tempaltes, TEI-Stylesheets.

build

We are using ant for building packages. Here we provide artifacts for download. In these packages we include the complete database and all packages required.

Dialogo-Medieval

Our first sample customization. It was created following the steps described earlier:

• Forking a project via Lab frontend (like the sample project to be found in the basic installation)
• Forking the SADE git project
• Rename the repo so it can be moved to the SADE group later (optional)
• Overriding the git files with the forked development
• Pushing to remote

Only one step is needed to hook in the deployment: Within the build repo just duplicate generic.xml and generic.build.properties and edit the git repo of the SADE package. By calling ant -f YOURBUILDFILE.xml the complete instance will be prepared.

Step by step we will extent the documentation.