Hydrilla

# Hydrilla on-disk data format

This page explains the upcoming format for Hydrilla site content stored in the filesystem. It refers to the upcoming Hydrilla 0.2 release.

{{toc}}

## How Hydrilla loads content

Hydrilla expects a content directory to be specified in its configuration file (under the key "content-dir"). It then processes all its direct subdirectories. If given subdirectory contains an `index.json` file, Hydrilla loads it (smartly ignoring "//" comments in it) and collects the definitions of site resources, pattern->payload mappings and licenses in it.

## Format of an index.json

To understand the format, look into this example file with explanatory comments in it:

``` javascript

// SPDX-License-Identifier: CC0-1.0

// Copyright (C) 2021 Wojtek Kosior

// Available under the terms of Creative Commons Zero v1.0 Universal.

// This is an example index.json file describing Hydrilla site content. As you

// can see, for storing site content information Hydrilla utilizes JSON with an

// additional extension in the form of '//' comments support. Hydrilla shall

// look into each direct subdirectory of the content directory passed to it

// (via a cofig file option). If such subsirectory contains an index.json file,

// Hydrilla shall process it.

// An index.json file conveys definitions of site resources, pattern->payload

// mappings and licenses thereof. The definitions may reference files under

// index.json's containing directory, using relative paths. This is how scripts,

// license texts, etc. are included. Unix paths (using '/' as separator) are

// assumed. It is not allowed for an index.json file to reference files outside

// its directory.

// Certain objects are allowed to contain a "comment" field. Although '//'

// comments can be used in index.json files, they will be stripped when the file

// is processed. If a comment should be included in the JSON definitions served

// by Hydrilla API, it should be put in a "comment" field of the proper object.

// Various kinds of objects contain version information. Version is always an

// array of integers, with major version number being the first array item. When

// applicable, a version is accompanied by a revision field which contains a

// positive integer. If versions specified by arrays of different length need to

// be compared, the shorter array gets padded with zeroes on the right. This

// means that for example version 1.3 could be given as both [1, 3] and

// [1, 3, 0, 0] (aka 1.3.0.0) and either would mean the same.

{

    // Once our json schema changes, this version will change. Our software will

    // be able to handle both current and older formats thanks to this

    // information present in every index.json file. Different schema versions

    // are always incompatible (e.g. a Hydrilla instance that understands schema

    // version 0.2.0.0 will not understand version 0.2.0.1). Schemas that are

    // backwards-compatible will be denoted by a different revision.

    // We will try to make schema version match the version of Hydrilla software

    // that introduced it.

    "schema_version": [0, 2],

    "schema_revision": 1,

    // Copyright of this json file. It's a list of copyright holder information

    // objects. Alternatively, "auto" can be used to make Hydrilla attempt to

    // extract copyright info from the comment at the beginning of the file.

    "copyright":  [

	// There can be multiple entries, one for each co-holder of the

	// copyright.

	{

	    // There can also be multiple years, like ["2021","2023-2024"].

	    "years": ["2021"],

	    // Name of the copyright holder. Depending on the situation it can

	    // be just the first name, name+surname, a company name, a

	    // pseudonym, etc.

	    "holder": "Wojtek Kosior"

	}

    ],

    // License of this json file. Identifier has to be known to Hydrilla. Can

    // be defined either in the same or another index.json file as a "license"

    // item. It is possible to specify license combinations, like:

    // [["Expat", "and", "Apache-2.0"], "or", "GPL-3.0-only"]

    // Alternatively, "auto" can be used to make Hydrilla attempt to extract

    // copyright info from this file's SPDX license identifier.

    "licenses": "CC0-1.0",

    // Where this software/work initially comes from. In some cases (i.e. when

    // the developer of content is also the one who packages it for Hydrilla)

    // this might be the same as "package_url".

    "upstream_url": "https://git.koszko.org/pydrilla/tree/example_content/hello",

    // Where sources for the packaging of this content can be found.

    "package_url": "https://git.koszko.org/pydrilla/tree/example_content/hello",

    // Additional "comment" field can be used if needed.

    // "comment": ""

    // List of actual site resources, pattern->payload mappings and licenses.

    // Each of them is represented by an object. Meta-sites and replacement site

    // interfaces will also belong here once they get implemented.

    "definitions": [

	{

	    // Value of "type" can currently be one of: "resource", "license"

	    // and "mapping". The one we have here, "resource", defines a list

	    // of injectable scripts that can be used as a payload or as a

	    // dependency of another "resource". In the future CSS style sheets

	    // and WASM modules will also be composite parts of a "resource" as

	    // scripts are now.

	    "type": "resource",

	    // Used when referring to this resource in "dependencies" list of

	    // another resource or in "payload" field of a mapping. Should

	    // be consize and can only use a restricted set of characters. It

	    // has to match: [-0-9a-zA-Z]

	    "identifier": "helloapple",

	    // "long_name" should be used to specify a user-friendly alternative

	    // to an identifier. It should generally not collide with a long

	    // name of some resource with a different uuid and also shouldn't

	    // change in-between versions of the same resource, although

	    // exceptions to both rules might be considered. Long name is

	    // allowed to contain arbitrary unicode characters (within reason!).

	    "long_name": "Hello Apple",

	    // Different versions (e.g. 1.0 and 1.3) of the same resource can be

	    // defined in separate index.json files. This makes it easy to

	    // accidently cause an identifier clash. To help detect it, we

	    // require that each resource has a uuid associated with it. Attempt

	    // to define multiple resources with the same identifier and

	    // different uuids will result in an error being reported. Defining

	    // multiple resources with different identifiers and the same uuid

	    // is disallowed for now (it may be later permitted if we consider

	    // it good for some use-case).

	    "uuid": "a6754dcb-58d8-4b7a-a245-24fd7ad4cd68",

	    // Version should match the upstream version of the resource (e.g. a

	    // version of javascript library). Revision number starts as 1 for

	    // each new resource version and gets incremented by 1 each time a

	    // modification to the packaging of this version is done. Hydrilla

	    // will allow multiple definitions of the same resource to load, as

	    // long as their versions differ. Thanks to the "version" and

	    // "revision" fields, clients will know they have to update certain

	    // resource after it has been updated. If multiple definitions of

	    // the same version of given resource are provided, an error is

	    // generated (even if those definitions differ by revision number).

	    "version": [2021, 11, 10],

	    "revision": 1,

	    // A short, meaningful description of what the resource is and/or

	    // what it does.

	    "description": "greets an apple",

	    // If needed, a "comment" field can be added to provide some

	    // additional information.

	    // "comment": "this resource something something",

	    // One should specify the copyright and licensing terms of the

	    // entire package. The format is the same as when specifying these

	    // for the index.json file, except "auto" cannot be used.

	    "copyright": [{"years": ["2021"], "holder": "Wojtek Kosior"}],

	    "licenses": "CC0-1.0",

	    // Resource's "dependencies" array shall contain names of other

	    // resources that (in case of scripts at least) should get evaluated

	    // on a page before this resource's own scripts.

	    "dependencies": ["hello-message"],

	    // Array of javascript files that belong to this resource.

	    "scripts": [

		{

		    // Script name. It should also be a valid file path.

		    "name": "hello.js",

		    // Copyright and license info of a script file can be

		    // specified using the same format as in the case of the

		    // index.json file itself. If "copyright" or "license" is

		    // not provided, Hydrilla assumes it to be the same as the

		    // value specified for the resource itself.

		    "copyright": "auto",

		    "licenses":  "auto"

		}, {

		    "name":   "bye.js"

		}

	    ]

	}, {

	    "type":       "resource",

	    "identifier": "hello-message",

	    "long_name":  "Hello Message",

	    "uuid":       "1ec36229-298c-4b35-8105-c4f2e1b9811e",

	    "version":     [2021, 11, 10],

	    "revision":    2,

	    "description": "define messages for saying hello and bye",

	    "copyright":   [{"years": ["2021"], "holder": "Wojtek Kosior"}],

	    "licenses":    "CC0-1.0",

	    // If "dependencies" is empty, it can also be omitted.

	    // "dependencies": [],

	    "scripts": [{"name": "message.js"}]

	}, {

	    "type": "mapping",

	    // Has similar function to resource's identifier. Should be consize

	    // and can only use a restricted set of characters. It has to match:

	    // [-0-9a-zA-Z]

	    // It can be the same as some resource identifier (those are

	    // different entities and are treated separately).

	    "identifier": "helloapple",

	    // "long name" and "uuid" have the same meaning as in the case of

	    // resources. Uuids of a resource and a mapping can technically be

	    // the same, but it is recommended to avoid even this kind of

	    // repetition.

	    "long_name": "Hello Apple",

	    "uuid": "54d23bba-472e-42f5-9194-eaa24c0e3ee7",

	    // "version" differs from its counterpart in resource in that it has

	    // no accompanying revision number.

	    "version": [2021, 11, 10],

	    // A short, meaningful description of what the mapping does.

	    "description": "causes apple to get greeted on Hydrillabugs issue tracker",

	    // A comment, if necessary.

	    // "comment": "blah blah because bleh"

	    // The "payloads" array specifies, which payloads are to be

	    // applied to which URLs.

	    "payloads": [

		{

		    // Should be a valid Haketilo URL pattern.

		    "pattern": "https://hydrillabugs.koszko.org/***",

		    // Should be the name of an existing resource. The resource

		    // may, but doesn't have to, be defined in the same

		    // index.json file.

		    "payload": "helloapple"

		},

		// More associations may follow.

		{

		    "pattern": "https://hachettebugs.koszko.org/***",

		    "payload": "helloapple"

		}

	    ]

	}, {

	    "type": "license",

	    // Will be used to refer to this license in other places. Should

	    // match the SPDX identifier if possible (despite that, please use

	    // "Expat" instead of "MIT" where possible). Unlike other definition

	    // types, "license" does not allow uuids to be used to avoid license

	    // id clashes. Any attempt to define multiple licenses with the same

	    // id will result in an error being reported.

	    "identifier": "CC0-1.0",

	    // This long name must also be unique among all license definitions.

	    "long_name": "Creative Commons Zero v1.0 Universal",

	    // We don't use "version" in license definitions. We do, however,

	    // use "revision" to indicate changes to the packaging of a license.

	    // Revision should be increased by 1 at each such change.

	    "revision": 2,

	    "legal_text": [

		// Legal text can be available in multiple forms. Usually just

		// plain .txt file is enough, though.

		{

		    // "format" should match an agreed-upon MIME type if

		    // possible.

		    "format": "text/plain",

		    // Value of "file" should be a path relative to the

		    // directory of index.json file.

		    "file":   "cc0.txt"

		}

		// If a markdown version of CC0 was provided, we could add this:

		// {

		//     "format": "text/markdown",

		//     "file": "cc0.md"

		// }

	    ]

	    // If needed, a "comment" field can be added to clarify something.

	    // For example, when definind "Expat" license we could add:

	    //

	    // "comment": "Expat license is the most common form of the license often called \"MIT\". Many other forms of \"MIT\" license exist. Here the name \"Expat\" is used to avoid ambiguity."

	    // If applicable, a "notice" can be included. It shall then be a

	    // path (relative to index.json) to a plain text file with that

	    // notice.

	    //

	    // "notice": "license-notice.txt"

	    //

	    // This is needed for example in case of GNU licenses (both with and

	    // without exceptions). For example,

	    // "GPL-3.0-or-later-with-html-exception" could have the following

	    // in its notice file:

	    //

	    // This program is free software: you can redistribute it and/or

	    // modify it under the terms of the GNU General Public License as

	    // published by the Free Software Foundation, either version 3 of

	    // the License, or (at your option) any later version.

	    //

	    // This program is distributed in the hope that it will be useful,

	    // but WITHOUT ANY WARRANTY; without even the implied warranty of

	    // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

	    // GNU General Public License for more details.

	    //

	    // As a special exception to the GPL, any HTML file which merely

	    // makes function calls to this code, and for that purpose

	    // includes it by reference shall be deemed a separate work for

	    // copyright law purposes.  If you modify this code, you may extend

	    // this exception to your version of the code, but you are not

	    // obligated to do so.  If you do not wish to do so, delete this

	    // exception statement from your version.

	    //

	    // You should have received a copy of the GNU General Public License

	    // along with this program.  If not, see

	    // <https://www.gnu.org/licenses/>.

	}

    ]

}

```