Hydrilla

# Hydrilla on-disk data format

This page explains the upcoming format for Hydrilla site content stored in the filesystem. It refers to the upcoming Hydrilla 0.2 release.

{{toc}}

## How Hydrilla loads content

Hydrilla expects a content directory to be specified in its configuration file (under the key "content-dir"). It then processes all its direct subdirectories. If given subdirectory contains an `index.json` file, Hydrilla loads it (smartly ignoring '//' comments in it) and collects the definitions of site resources, pattern->payload mappings and licenses in it.

## Format of an index.json

To understand the format, look into this example file with explanatory comments in it:

``` javascript

// SPDX-License-Identifier: CC0-1.0

// Copyright (C) 2021 Wojtek Kosior

// Available under the terms of Creative Commons Zero v1.0 Universal.

// This is an example index.json file describing Hydrilla site content. As you

// can see, for storing site content information Hydrilla utilizes JSON with an

// additional extension in the form of '//' comments support. Hydrilla shall

// look into each direct subdirectory of the content directory passed to it

// (via a cofig file option). If such subsirectory contains an index.json file,

// Hydrilla shall process it.

// An index.json file conveys definitions of site resources, pattern->payload

// mappings and licenses thereof. The definitions may reference files under

// index.json's containing directory, using relative paths. This is how scripts,

// license texts, etc. are included. Unix paths (using '/' as separator) are

// assumed. It is not allowed for an index.json file to reference files outside

// its directory.

// Certain objects are allowed to contain a "comment" field. Although '//'

// comments can be used in index.json files, they will be stripped when the file

// is processed. If a comment should be included in the JSON definitions served

// by Hydrilla API, it should be put in a "comment" field of the proper object.

// Various kinds of objects contain version information. Version is always an

// array of integers, with major version number being the first array item. When

// applicable, a version might be accompanied by a revision field which contains

// a positive integer and is optional (defaults to 1 if not specified). If

// versions specified by arrays of different length need to be compared, the

// shorter array gets padded with zeroes on the right. This means that for

// example version 1.3 could be given as both [1, 3] and [1, 3, 0, 0]

// (aka 1.3.0.0) and either would mean the same.

{

    // Once our json schema changes, this version will change. Our software will

    // be able to handle both current and older formats thanks to this

    // information present in every index.json file. Different schema versions are always incompatible (e.g. a Hydrilla instance that understands schema version 0.2.0.0 will not understand version 0.2.0.1). Schemas that are backwards-compatible will be denoted by a different revision.

    // We will try to make schema version match the version of Hydrilla software that introduced it.

    "schema_version": [0, 2],

    // 1 is the default revision. We could have as well just skipped this field.

    "schema_revision": 1,

    // Copyright of this json file. It's a list of copyright holder information

    // objects. Alternatively, "auto" can be used to make Hydrilla attempt to

    // extract copyright info from the comment at the beginning of the file.

    "copyright":  [

	// There can be multiple entries, one for each co-holder of the

	// copyright.

	{

	    // There can also be multiple years, like ["2021","2023-2024"].

	    "years": ["2021"],

	    // Name of the copyright holder. Depending on the situation it can

	    // be just the first name, name+surname, a company name, a

	    // pseudonym, etc.

	    "holder": "Wojtek Kosior"

	}

    ],

    // License of this json file. Identifier has to be known to Hydrilla. Can

    // be defined either in the same or another index.json file as a "license"

    // item. It is possible to specify license combinations, like:

    // [["Expat", "and", "Apache-2.0"], "or", "GPL-3.0-only"]

    // Alternatively, "auto" can be used to make Hydrilla attempt to extract

    // copyright info from this file's SPDX license identifier.

    "licenses": "CC0-1.0",

    // Where this software/work initially comes from. In some cases (i.e. when

    // the developer of content is also the one who packages it for Hydrilla)

    // this might be the same as "package_url".

    "upstream_url": "https://git.koszko.org/pydrilla/tree/example_content/hello",

    // Where sources for the packaging of this content can be found.

    "package_url": "https://git.koszko.org/pydrilla/tree/example_content/hello",

    // Additional "comment" field can be used if needed.

    // "comment": ""

    // List of actual site resources, pattern->payload mappings and licenses.

    // Each of them is represented by an object. Meta-sites and replacement site

    // interfaces will also belong here once they get implemented.

    "definitions": [

	{

	    // Value of "type" can currently be one of: "resource", "license"

	    // and "mapping". The one we have here, "resource", defines a list

	    // of injectable scripts that can be used as a payload or as a

	    // dependency of another "resource". In the future CSS style sheets

	    // and WASM modules will also be composite parts of a "resource" as

	    // scripts are now.

	    "type": "resource",

	    // Used when referring to this resource in "dependencies" list of

	    // another resource or in "payload" field of a mapping. Should

	    // be consize and can only use a restricted set of characters. It

	    // has to match: [-0-9a-zA-Z]

	    "identifier": "helloapple",

	    // "long_name" should be used to specify a user-friendly alternative

	    // to an identifier. It should not collide with a long name of some

	    // resource with a different uuid. However, it can change in-between

	    // versions of the same resource and is allowed to contain arbitrary

	    // unicode characters (within reason!).

	    "long_name": "Hello Apple",

	    // Different versions of the same resource can be defined in

	    // separate index.json files. This makes it easy to accidently cause

	    // an identifier clash. To help detect it, we require that each

	    // resource has a uuid associated with it. Attempt to define

	    // multiple resources with the same identifier and different uuids

	    // will result in an error being reported. Defining multiple

	    // resources with different identifiers and the same uuid is

	    // disallowed for now (it may be later permitted if we consider it

	    // good for some use-case).

	    "uuid": "a6754dcb-58d8-4b7a-a245-24fd7ad4cd68",

	    // "old_uuids" field can be used to indicate that some specific

	    // resource identifier collision is allowed, for example when some

	    // resource (e.g. a javascript library) changes its versioning

	    // schema. All definitions with one of the old uuids will be treated

	    // as less up-to-date versions, overriding the "version" field

	    // values comparison result.

	    "old_uuids": ["ad38dc5e-30b7-492d-9290-6c3ca658f1f3"],

	    // Version should match the upstream version of the resource (e.g. a

	    // version of javascript library). Revision number starts as 1 for

	    // each new resource version and gets incremented by 1 each time a

	    // modification to the packaging of this version is done.

	    "version": [2021, 11, 10],

	    // Specifying no revision is the same as specifying:

	    // "revision": 1,

	    // A short, meaningful description of what the resource is and/or

	    // what it does.

	    "description": "greets an apple",

	    // If needed, a "comment" field can be added to provide some

	    // additional information.

	    // "comment": "this resource something something",

	    // One should specify the copyright and licensing terms of the

	    // entire package. The format is the same as when specifying these

	    // for the index.json file, except "auto" cannot be used.

	    "copyright": [{"years": ["2021"], "holder": "Wojtek Kosior"}],

	    "licenses": "CC0-1.0",

	    // Resource's "dependencies" array shall contain names of other

	    // resources that (in case of scripts at least) should get evaluated

	    // on a page before this resource's own scripts.

	    "dependencies": ["hello-message"],

	    // Array of javascript files that belong to this resource.

	    "scripts": [

		{

		    // Script name. It should also be a valid file path.

		    "name": "hello.js",

		    // Copyright and license info of a script file can be

		    // specified using the same format as in the case of the

		    // index.json file itself. If "copyright" or "license" is

		    // not provided, Hydrilla assumes it to be the same as the

		    // value specified for the resource itself.

		    "copyright": "auto",

		    "licenses":  "auto"

		}, {

		    "name":   "bye.js"

		}

	    ]

	}, {

	    "type":       "resource",

	    "identifier": "hello-message",

	    "long_name":  "Hello Message",

	    "uuid":       "1ec36229-298c-4b35-8105-c4f2e1b9811e",

	    // If "old_uuids" is empty, it can as well be omitted.

	    // "old_uuids": [],

	    "version":     [2021, 11, 10],

	    "revision":    2,

	    "description": "define messages for saying hello and bye",

	    "copyright":   [{"years": ["2021"], "holder": "Wojtek Kosior"}],

	    "licenses":    "CC0-1.0",

	    // If "dependencies" is empty, it can also be omitted.

	    // "dependencies": [],

	    "scripts": [{"name": "message.js"}]

	}, {

	    "type": "mapping",

	    // Has similar function to resource's identifier. Should be consize

	    // and can only use a restricted set of characters. It has to match:

	    // [-0-9a-zA-Z]

	    // It can be the same as some resource identifier (those are

	    // different entities and are treated separately).

	    "identifier": "helloapple",

	    // "long name", "uuid" and "old_uuids" have the same meaning as in

	    // the case of resources. Uuids of a resource and a mapping can

	    // technically be the same, but it is recommended to avoid even this

	    // kind of repetition.

	    "long_name": "Hello Apple",

	    "uuid": "54d23bba-472e-42f5-9194-eaa24c0e3ee7",

	    "old_uuids": ["0525824d-f6e2-4ee2-8ac6-401e9dc79cdf"],

	    // "version" differs from its counterpart in resource in that it has

	    // no accompanying revision number.

	    "version": [2021, 11, 10],

	    // A short, meaningful description of what the mapping does.

	    "description": "causes apple to get greeted on Hydrillabugs issue tracker",

	    // A comment, if necessary.

	    // "comment": "blah blah because bleh"

	    // The "injections" array specifies, which payloads are to be

	    // applied to which URLs.

	    "injctions": [

		{

		    // Should be a valid Haketilo URL pattern.

		    "pattern": "https://hydrillabugs.koszko.org/***",

		    // Should be the name of an existing resource. The resource

		    // may, but doesn't have to, be defined in the same

		    // index.json file.

		    "payload": "helloapple"

		},

		// More associations may follow.

		{

		    "pattern": "https://hachettebugs.koszko.org/***",

		    "payload": "helloapple"

		}

	    ]

	}, {

	    "type": "license",

	    // Will be used to refer to this license in other places. Should

	    // match the SPDX identifier if possible (despite that, please use

	    // "Expat" instead of "MIT" where possible). Unlike other definition

	    // types, "license" does not allow uuids to be used to avoid license

	    // id clashes. Any attempt to define multiple licenses with the same

	    // id will result in an error being reported.

	    "identifier": "CC0-1.0",

	    // This long name must also be unique among all license definitions.

	    "long_name": "Creative Commons Zero v1.0 Universal",

	    "legal_text": [

		// Legal text can be available in multiple forms. Usually just

		// plain .txt file is enough, though.

		{

		    // "format" should match an agreed-upon MIME type if

		    // possible.

		    "format": "text/plain",

		    // Value of "file" should be a path relative to the

		    // directory of index.json file.

		    "file":   "cc0.txt"

		}

		// If a markdown version of CC0 was provided, we could add this:

		// {

		//     "format": "text/markdown",

		//     "file": "cc0.md"

		// }

	    ]

	    // If needed, a "comment" field can be added to clarify something.

	    // For example, when definind "Expat" license we could add:

	    //

	    // "comment": "Expat license is the most common form of the license often called \"MIT\". Many other forms of \"MIT\" license exist. Here the name \"Expat\" is used to avoid ambiguity."

	    // If applicable, a "notice" can be included. It shall then be a

	    // path (relative to index.json) to a plain text file with that

	    // notice.

	    //

	    // "notice": "license-notice.txt"

	    //

	    // This is needed for example in case of GNU licenses (both with and

	    // without exceptions). For example,

	    // "GPL-3.0-or-later-with-html-exception" could have the following

	    // in its notice file:

	    //

	    // This program is free software: you can redistribute it and/or

	    // modify it under the terms of the GNU General Public License as

	    // published by the Free Software Foundation, either version 3 of

	    // the License, or (at your option) any later version.

	    //

	    // This program is distributed in the hope that it will be useful,

	    // but WITHOUT ANY WARRANTY; without even the implied warranty of

	    // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

	    // GNU General Public License for more details.

	    //

	    // As a special exception to the GPL, any HTML file which merely

	    // makes function calls to this code, and for that purpose

	    // includes it by reference shall be deemed a separate work for

	    // copyright law purposes.  If you modify this code, you may extend

	    // this exception to your version of the code, but you are not

	    // obligated to do so.  If you do not wish to do so, delete this

	    // exception statement from your version.

	    //

	    // You should have received a copy of the GNU General Public License

	    // along with this program.  If not, see

	    // <https://www.gnu.org/licenses/>.

	}

    ]

}

```