Hydrilla on-disk data format¶

This page explains the upcoming format for Hydrilla site content stored in the filesystem. It refers to the upcoming Hydrilla 0.2 release.

• Table of contents
• Hydrilla on-disk data format
  • How Hydrilla loads content
  • Format of an index.json

How Hydrilla loads content¶

Hydrilla expects a content directory to be specified in its configuration file (under the key "content-dir"). It then processes all its direct subdirectories. If given subdirectory contains an index.json file, Hydrilla loads it (smartly ignoring '//' comments in it) and collects the definitions of site resources, pattern->payload mappings and licenses in it.

Format of an index.json¶

To understand the format, look into this example file with explanatory comments in it:

// SPDX-License-Identifier: CC0-1.0

// Copyright (C) 2021 Wojtek Kosior
// Available under the terms of Creative Commons Zero v1.0 Universal.

// This is an example index.json file describing Hydrilla site content. As you
// can see, for storing site content information Hydrilla utilizes JSON with an
// additional extension in the form of '//' comments support. Hydrilla shall
// look into each direct subdirectory of the content directory passed to it
// (via a cofig file option). If such subsirectory contains an index.json file,
// Hydrilla shall process it.

// An index.json file conveys definitions of site resources, pattern->payload
// mappings and licenses thereof. The definitions may reference files under
// index.json's containing directory, using relative paths. This is how scripts,
// license texts, etc. are included. Unix paths (using '/' as separator) are
// assumed. It is not allowed for an index.json file to reference files outside
// its directory.

// Certain objects are allowed to contain a "comment" field. Although '//'
// comments can be used in index.json files, they will be stripped when the file
// is processed. If a comment should be included in the JSON definitions served
// by Hydrilla API, it should be put in a "comment" field of the proper object.

// Various kinds of objects contain version information. Version is always an
// array of integers, with major version number being the first array item. When
// applicable, a version is accompanied by a revision field which contains a
// positive integer. If versions specified by arrays of different length need to
// be compared, the shorter array gets padded with zeroes on the right. This
// means that for example version 1.3 could be given as both [1, 3] and
// [1, 3, 0, 0] (aka 1.3.0.0) and either would mean the same.

{
    // Once our json schema changes, this version will change. Our software will
    // be able to handle both current and older formats thanks to this
    // information present in every index.json file. Different schema versions
    // are always incompatible (e.g. a Hydrilla instance that understands schema
    // version 0.2.0.0 will not understand version 0.2.0.1). Schemas that are
    // backwards-compatible will be denoted by a different revision.
    // We
    // will try to make schema version match the version of Hydrilla software
    // that introduced it.
    "schema_version": [0, 2],
    "schema_revision": 1,

    // Copyright of this json file. It's a list of copyright holder information
    // objects. Alternatively, "auto" can be used to make Hydrilla attempt to
    // extract copyright info from the comment at the beginning of the file.
    "copyright":  [
    // There can be multiple entries, one for each co-holder of the
    // copyright.
    {
        // There can also be multiple years, like ["2021","2023-2024"].
        "years": ["2021"],
        // Name of the copyright holder. Depending on the situation it can
        // be just the first name, name+surname, a company name, a
        // pseudonym, etc.
        "holder": "Wojtek Kosior"
    }
    ],

    // License of this json file. Identifier has to be known to Hydrilla. Can
    // be defined either in the same or another index.json file as a "license"
    // item. It is possible to specify license combinations, like:
    // [["Expat", "and", "Apache-2.0"], "or", "GPL-3.0-only"]
    // Alternatively, "auto" can be used to make Hydrilla attempt to extract
    // copyright info from this file's SPDX license identifier.
    "licenses": "CC0-1.0",

    // Where this software/work initially comes from. In some cases (i.e. when
    // the developer of content is also the one who packages it for Hydrilla)
    // this might be the same as "package_url".
    "upstream_url": "https://git.koszko.org/pydrilla/tree/example_content/hello",

    // Where sources for the packaging of this content can be found.
    "package_url": "https://git.koszko.org/pydrilla/tree/example_content/hello",

    // Additional "comment" field can be used if needed.
    // "comment": ""

    // List of actual site resources, pattern->payload mappings and licenses.
    // Each of them is represented by an object. Meta-sites and replacement site
    // interfaces will also belong here once they get implemented.
    "definitions": [
    {
        // Value of "type" can currently be one of: "resource", "license"
        // and "mapping". The one we have here, "resource", defines a list
        // of injectable scripts that can be used as a payload or as a
        // dependency of another "resource". In the future CSS style sheets
        // and WASM modules will also be composite parts of a "resource" as
        // scripts are now.
        "type": "resource",

        // Used when referring to this resource in "dependencies" list of
        // another resource or in "payload" field of a mapping. Should
        // be consize and can only use a restricted set of characters. It
        // has to match: [-0-9a-zA-Z]
        "identifier": "helloapple",

        // "long_name" should be used to specify a user-friendly alternative
        // to an identifier. It should generally not collide with a long
        // name of some resource with a different uuid and also shouldn't
        // change in-between versions of the same resource, although
        // exceptions to both rules might be considered. Long name is
        // allowed to contain arbitrary unicode characters (within reason!).
        "long_name": "Hello Apple",

        // Different versions (e.g. 1.0 and 1.3) of the same resource can be
        // defined in separate index.json files. This makes it easy to
        // accidently cause an identifier clash. To help detect it, we
        // require that each resource has a uuid associated with it. Attempt
        // to define multiple resources with the same identifier and
        // different uuids will result in an error being reported. Defining
        // multiple resources with different identifiers and the same uuid
        // is disallowed for now (it may be later permitted if we consider
        // it good for some use-case).
        "uuid": "a6754dcb-58d8-4b7a-a245-24fd7ad4cd68",

        // Version should match the upstream version of the resource (e.g. a
        // version of javascript library). Revision number starts as 1 for
        // each new resource version and gets incremented by 1 each time a
        // modification to the packaging of this version is done.
        "version": [2021, 11, 10],
        "revision": 1,

        // A short, meaningful description of what the resource is and/or
        // what it does.
        "description": "greets an apple",

        // If needed, a "comment" field can be added to provide some
        // additional information.
        // "comment": "this resource something something",

        // One should specify the copyright and licensing terms of the
        // entire package. The format is the same as when specifying these
        // for the index.json file, except "auto" cannot be used.
        "copyright": [{"years": ["2021"], "holder": "Wojtek Kosior"}],
        "licenses": "CC0-1.0",

        // Resource's "dependencies" array shall contain names of other
        // resources that (in case of scripts at least) should get evaluated
        // on a page before this resource's own scripts.
        "dependencies": ["hello-message"],

        // Array of javascript files that belong to this resource.
        "scripts": [
        {
            // Script name. It should also be a valid file path.
            "name": "hello.js",
            // Copyright and license info of a script file can be
            // specified using the same format as in the case of the
            // index.json file itself. If "copyright" or "license" is
            // not provided, Hydrilla assumes it to be the same as the
            // value specified for the resource itself.
            "copyright": "auto",
            "licenses":  "auto"
        }, {
            "name":   "bye.js"
        }
        ]
    }, {
        "type":       "resource",
        "identifier": "hello-message",
        "long_name":  "Hello Message",
        "uuid":       "1ec36229-298c-4b35-8105-c4f2e1b9811e",
        "version":     [2021, 11, 10],
        "revision":    2,
        "description": "define messages for saying hello and bye",
        "copyright":   [{"years": ["2021"], "holder": "Wojtek Kosior"}],
        "licenses":    "CC0-1.0",
        // If "dependencies" is empty, it can also be omitted.
        // "dependencies": [],
        "scripts": [{"name": "message.js"}]
    }, {
        "type": "mapping",

        // Has similar function to resource's identifier. Should be consize
        // and can only use a restricted set of characters. It has to match:
        // [-0-9a-zA-Z]
        // It can be the same as some resource identifier (those are
        // different entities and are treated separately).
        "identifier": "helloapple",

        // "long name" and "uuid" have the same meaning as in the case of
        // resources. Uuids of a resource and a mapping can technically be
        // the same, but it is recommended to avoid even this kind of
        // repetition.
        "long_name": "Hello Apple",
        "uuid": "54d23bba-472e-42f5-9194-eaa24c0e3ee7",

        // "version" differs from its counterpart in resource in that it has
        // no accompanying revision number.
        "version": [2021, 11, 10],

        // A short, meaningful description of what the mapping does.
        "description": "causes apple to get greeted on Hydrillabugs issue tracker",

        // A comment, if necessary.
        // "comment": "blah blah because bleh"

        // The "injections" array specifies, which payloads are to be
        // applied to which URLs.
        "injctions": [
        {
            // Should be a valid Haketilo URL pattern.
            "pattern": "https://hydrillabugs.koszko.org/***",
            // Should be the name of an existing resource. The resource
            // may, but doesn't have to, be defined in the same
            // index.json file.
            "payload": "helloapple"
        },
        // More associations may follow.
        {
            "pattern": "https://hachettebugs.koszko.org/***",
            "payload": "helloapple"
        }
        ]
    }, {
        "type": "license",

        // Will be used to refer to this license in other places. Should
        // match the SPDX identifier if possible (despite that, please use
        // "Expat" instead of "MIT" where possible). Unlike other definition
        // types, "license" does not allow uuids to be used to avoid license
        // id clashes. Any attempt to define multiple licenses with the same
        // id will result in an error being reported.
        "identifier": "CC0-1.0",

        // This long name must also be unique among all license definitions.
        "long_name": "Creative Commons Zero v1.0 Universal",

        "legal_text": [
        // Legal text can be available in multiple forms. Usually just
        // plain .txt file is enough, though.
        {
            // "format" should match an agreed-upon MIME type if
            // possible.
            "format": "text/plain",
            // Value of "file" should be a path relative to the
            // directory of index.json file.
            "file":   "cc0.txt"
        }
        // If a markdown version of CC0 was provided, we could add this:
        // {
        //     "format": "text/markdown",
        //     "file": "cc0.md"
        // }
        ]

        // If needed, a "comment" field can be added to clarify something.
        // For example, when definind "Expat" license we could add:
        //
        // "comment": "Expat license is the most common form of the license often called \"MIT\". Many other forms of \"MIT\" license exist. Here the name \"Expat\" is used to avoid ambiguity."

        // If applicable, a "notice" can be included. It shall then be a
        // path (relative to index.json) to a plain text file with that
        // notice.
        //
        // "notice": "license-notice.txt"
        //
        // This is needed for example in case of GNU licenses (both with and
        // without exceptions). For example,
        // "GPL-3.0-or-later-with-html-exception" could have the following
        // in its notice file:
        //
        // This program is free software: you can redistribute it and/or
        // modify it under the terms of the GNU General Public License as
        // published by the Free Software Foundation, either version 3 of
        // the License, or (at your option) any later version.
        //
        // This program is distributed in the hope that it will be useful,
        // but WITHOUT ANY WARRANTY; without even the implied warranty of
        // MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
        // GNU General Public License for more details.
        //
        // As a special exception to the GPL, any HTML file which merely
        // makes function calls to this code, and for that purpose
        // includes it by reference shall be deemed a separate work for
        // copyright law purposes.  If you modify this code, you may extend
        // this exception to your version of the code, but you are not
        // obligated to do so.  If you do not wish to do so, delete this
        // exception statement from your version.
        //
        // You should have received a copy of the GNU General Public License
        // along with this program.  If not, see
        // <https://www.gnu.org/licenses/>.
    }
    ]
}