Hydrilla source package format » History » Version 3
koszko, 02/02/2022 11:28 AM
update source package index.json format and add a JSON schema
| 1 | 1 | koszko | # Hydrilla source package format |
|---|---|---|---|
| 2 | |||
| 3 | 3 | koszko | This page explains the upcoming format for source packages used to build Hydrilla site content. It refers to the upcoming Hydrilla 1.0 release and is subject to change before the actual release. |
| 4 | 1 | koszko | |
| 5 | You might also want to read about the data format of [[Hydrilla on-disk data format| built Hydrilla resources]]. |
||
| 6 | |||
| 7 | ## How Hydrilla packages are built |
||
| 8 | Hydrilla package builder expects a source package directory and a destination directory to be specified on the command line. If source package directory is missing, current directory is used as default. |
||
| 9 | |||
| 10 | ## Desination directory |
||
| 11 | The destination directory can be considered the same as [[Hydrilla on-disk data format#How-Hydrilla-loads-content|Hydrilla's content directory]] - built artifacts will be written in conformance with Hydrilla data format. |
||
| 12 | |||
| 13 | ## Source package directory |
||
| 14 | Source package directory is expectes to contain an `index.json` file, Hydrilla loads it (smartly ignoring "//" comments in it) and collects the definitions of site resources and pattern->payload mappings in it. |
||
| 15 | |||
| 16 | ## Format of an index.json |
||
| 17 | 3 | koszko | ### Example |
| 18 | 1 | koszko | To understand the format, look into this example file with explanatory comments in it: |
| 19 | |||
| 20 | ``` javascript |
||
| 21 | // SPDX-License-Identifier: CC0-1.0 |
||
| 22 | |||
| 23 | // Copyright (C) 2021 Wojtek Kosior |
||
| 24 | // Available under the terms of Creative Commons Zero v1.0 Universal. |
||
| 25 | |||
| 26 | // This is an example index.json file describing Hydrilla site content. As you |
||
| 27 | // can see, for storing site content information Hydrilla utilizes JSON with an |
||
| 28 | // additional extension in the form of '//' comments support. |
||
| 29 | |||
| 30 | // An index.json file conveys definitions of site resources and pattern->payload |
||
| 31 | // mappings. The definitions may reference files under index.json's containing |
||
| 32 | // directory, using relative paths. This is how scripts, license texts, etc. are |
||
| 33 | // included. |
||
| 34 | // File reference always takes the form of an object with "file" property |
||
| 35 | // specifying path to the file. In certain contexts additional properties may be |
||
| 36 | // allowed or required. Unix paths (using '/' as separator) are assumed. It is |
||
| 37 | // not allowed for an index.json file to reference files outside its directory. |
||
| 38 | |||
| 39 | // Certain objects are allowed to contain a "comment" field. Although '//' |
||
| 40 | // comments can be used in index.json files, they will not be included in |
||
| 41 | // generated JSON definitions. If a comment should be included in the |
||
| 42 | // definitions served by Hydrilla API, it should be put in a "comment" field of |
||
| 43 | // the proper object. |
||
| 44 | |||
| 45 | // Unknown object properties will be ignored. This is for compatibility with |
||
| 46 | // possible future revisions of the format. |
||
| 47 | |||
| 48 | // Various kinds of objects contain version information. Version is always an |
||
| 49 | // array of integers, with major version number being the first array item. When |
||
| 50 | // applicable, a version is accompanied by a revision field which contains a |
||
| 51 | // positive integer. If versions specified by arrays of different length need to |
||
| 52 | // be compared, the shorter array gets padded with zeroes on the right. This |
||
| 53 | // means that for example version 1.3 could be given as both [1, 3] and |
||
| 54 | // [1, 3, 0, 0] (aka 1.3.0.0) and either would mean the same. |
||
| 55 | |||
| 56 | { |
||
| 57 | // Once our index.json schema changes, this field's value will change. Our |
||
| 58 | // software will be able to handle both current and older formats thanks to |
||
| 59 | 3 | koszko | // this information present in every index.json file. Schemas that differ by |
| 60 | // the first (major) number are always incompatible (e.g. a Hydrilla builder |
||
| 61 | // instance released at the time of 1.2 being the most recent schema version |
||
| 62 | // will not understand version 2.0). |
||
| 63 | // Schemas that are backwards-compatible will have the same major number |
||
| 64 | // and might differ by the second (minor) version number. The third (patch) |
||
| 65 | // and subsequent numbers are being ignored right now. |
||
| 66 | 1 | koszko | "source_schema_version": [1], |
| 67 | |||
| 68 | // Used when referring to this source package. Should be consize, unique |
||
| 69 | // (among other source package names) and can only use a restricted set of |
||
| 70 | // characters. It has to match: [-0-9a-z.]+ |
||
| 71 | // If source name is lacking, Hydrilla package builder will use index.json's |
||
| 72 | // directory name instead, with all disallowed characters converted to |
||
| 73 | // hyphens ("-"). |
||
| 74 | "source_name": "hello", |
||
| 75 | |||
| 76 | // Copyright of this source package. Should list files that contain |
||
| 77 | // copyright information regarding this source package as well as texts of |
||
| 78 | // licenses used. Although no specific format of these files is mandated, it |
||
| 79 | // is recommended to make each source package REUSE-compliant, generate an |
||
| 80 | // spdx report for it as `report.spdx` and list this report together with |
||
| 81 | // all license files here. |
||
| 82 | "copyright": [ |
||
| 83 | 3 | koszko | {"file": "report.spdx"}, |
| 84 | {"file": "LICENSES/CC0-1.0.txt"} |
||
| 85 | 1 | koszko | ], |
| 86 | |||
| 87 | // Where this software/work initially comes from. |
||
| 88 | "upstream_url": "https://git.koszko.org/pydrilla/tree/src/test/example_content/hello", |
||
| 89 | |||
| 90 | // Additional "comment" field can be used if needed. |
||
| 91 | // "comment": "" |
||
| 92 | |||
| 93 | // List of actual site resources and pattern->payload mappings. Each of them |
||
| 94 | // is represented by an object. Meta-sites and replacement site interfaces |
||
| 95 | // will also belong here once they get implemented. |
||
| 96 | "definitions": [ |
||
| 97 | 3 | koszko | { |
| 98 | // Value of "type" can currently be one of: "resource" and |
||
| 99 | // "mapping". The one we have here, "resource", defines a list |
||
| 100 | // of injectable scripts that can be used as a payload or as a |
||
| 101 | // dependency of another "resource". In the future CSS style sheets |
||
| 102 | // and WASM modules will also be composite parts of a "resource" as |
||
| 103 | // scripts are now. |
||
| 104 | "type": "resource", |
||
| 105 | 1 | koszko | |
| 106 | 3 | koszko | // Used when referring to this resource in "dependencies" list of |
| 107 | // another resource or in "payload" field of a mapping. Should |
||
| 108 | // be consize and can only use a restricted set of characters. It |
||
| 109 | // has to match: [-0-9a-z]+ |
||
| 110 | "identifier": "helloapple", |
||
| 111 | 1 | koszko | |
| 112 | 3 | koszko | // "long_name" should be used to specify a user-friendly alternative |
| 113 | // to an identifier. It should generally not collide with a long |
||
| 114 | // name of some resource with a different uuid and also shouldn't |
||
| 115 | // change in-between versions of the same resource, although |
||
| 116 | // exceptions to both rules might be considered. Long name is |
||
| 117 | // allowed to contain arbitrary unicode characters (within reason!). |
||
| 118 | "long_name": "Hello Apple", |
||
| 119 | 1 | koszko | |
| 120 | 3 | koszko | // Different versions (e.g. 1.0 and 1.3) of the same resource can be |
| 121 | // defined in separate index.json files. This makes it easy to |
||
| 122 | // accidently cause an identifier clash. To help detect it, we |
||
| 123 | // require that each resource has a uuid associated with it. Attempt |
||
| 124 | // to define multiple resources with the same identifier and |
||
| 125 | // different uuids will result in an error being reported. Defining |
||
| 126 | // multiple resources with different identifiers and the same uuid |
||
| 127 | // is disallowed for now (it may be later permitted if we consider |
||
| 128 | // it good for some use-case). |
||
| 129 | "uuid": "a6754dcb-58d8-4b7a-a245-24fd7ad4cd68", |
||
| 130 | 1 | koszko | |
| 131 | 3 | koszko | // Version should match the upstream version of the resource (e.g. a |
| 132 | // version of javascript library). Revision number starts as 1 for |
||
| 133 | // each new resource version and gets incremented by 1 each time a |
||
| 134 | // modification to the packaging of this version is done. Hydrilla |
||
| 135 | // will allow multiple definitions of the same resource to load, as |
||
| 136 | // long as their versions differ. Thanks to the "version" and |
||
| 137 | // "revision" fields, clients will know they have to update certain |
||
| 138 | // resource after it has been updated. If multiple definitions of |
||
| 139 | // the same version of given resource are provided, an error is |
||
| 140 | // generated (even if those definitions differ by revision number). |
||
| 141 | "version": [2021, 11, 10], |
||
| 142 | "revision": 1, |
||
| 143 | 1 | koszko | |
| 144 | 3 | koszko | // A short, meaningful description of what the resource is and/or |
| 145 | // what it does. |
||
| 146 | "description": "greets an apple", |
||
| 147 | 1 | koszko | |
| 148 | 3 | koszko | // If needed, a "comment" field can be added to provide some |
| 149 | // additional information. |
||
| 150 | // "comment": "this resource something something", |
||
| 151 | 1 | koszko | |
| 152 | 3 | koszko | // Resource's "dependencies" array shall contain names of other |
| 153 | // resources that (in case of scripts at least) should get evaluated |
||
| 154 | // on a page before this resource's own scripts. |
||
| 155 | "dependencies": ["hello-message"], |
||
| 156 | 1 | koszko | |
| 157 | 3 | koszko | // Array of javascript files that belong to this resource. |
| 158 | "scripts": [ |
||
| 159 | {"file": "hello.js"}, |
||
| 160 | {"file": "bye.js"} |
||
| 161 | ] |
||
| 162 | }, { |
||
| 163 | "type": "resource", |
||
| 164 | "identifier": "hello-message", |
||
| 165 | "long_name": "Hello Message", |
||
| 166 | "uuid": "1ec36229-298c-4b35-8105-c4f2e1b9811e", |
||
| 167 | "version": [2021, 11, 10], |
||
| 168 | "revision": 2, |
||
| 169 | "description": "define messages for saying hello and bye", |
||
| 170 | // If "dependencies" is empty, it can also be omitted. |
||
| 171 | // "dependencies": [], |
||
| 172 | "scripts": [{"file": "message.js"}] |
||
| 173 | }, { |
||
| 174 | "type": "mapping", |
||
| 175 | 1 | koszko | |
| 176 | 3 | koszko | // Has similar function to resource's identifier. Should be consize |
| 177 | // and can only use a restricted set of characters. It has to match: |
||
| 178 | // [-0-9a-z]+ |
||
| 179 | // It can be the same as some resource identifier (those are |
||
| 180 | // different entities and are treated separately). |
||
| 181 | "identifier": "helloapple", |
||
| 182 | 1 | koszko | |
| 183 | 3 | koszko | // "long name" and "uuid" have the same meaning as in the case of |
| 184 | // resources. Uuids of a resource and a mapping can technically be |
||
| 185 | // the same, but it is recommended to avoid even this kind of |
||
| 186 | // repetition. |
||
| 187 | "long_name": "Hello Apple", |
||
| 188 | "uuid": "54d23bba-472e-42f5-9194-eaa24c0e3ee7", |
||
| 189 | 1 | koszko | |
| 190 | 3 | koszko | // "version" differs from its counterpart in resource in that it has |
| 191 | // no accompanying revision number. |
||
| 192 | "version": [2021, 11, 10], |
||
| 193 | 1 | koszko | |
| 194 | 3 | koszko | // A short, meaningful description of what the mapping does. |
| 195 | "description": "causes apple to get greeted on Hydrillabugs issue tracker", |
||
| 196 | 1 | koszko | |
| 197 | 3 | koszko | // A comment, if necessary. |
| 198 | // "comment": "blah blah because bleh" |
||
| 199 | 1 | koszko | |
| 200 | 3 | koszko | // The "payloads" object specifies which payloads are to be applied |
| 201 | // to which URLs. |
||
| 202 | "payloads": { |
||
| 203 | // Each key should be a valid Haketilo URL pattern. |
||
| 204 | "https://hydrillabugs.koszko.org/***": { |
||
| 205 | // Should be the name of an existing resource. The resource |
||
| 206 | // may, but doesn't have to, be defined in the same |
||
| 207 | // index.json file. |
||
| 208 | "identifier": "helloapple" |
||
| 209 | 1 | koszko | }, |
| 210 | 3 | koszko | // More associations may follow. |
| 211 | "https://hachettebugs.koszko.org/***": { |
||
| 212 | "identifier": "helloapple" |
||
| 213 | } |
||
| 214 | } |
||
| 215 | } |
||
| 216 | 1 | koszko | ] |
| 217 | } |
||
| 218 | 3 | koszko | ``` |
| 219 | |||
| 220 | ### JSON schema |
||
| 221 | This is a Draft-07 JSON schema that an index.json file should conform to: |
||
| 222 | ``` javascript |
||
| 223 | { |
||
| 224 | "$schema": "https://json-schema.org/draft-07/schema", |
||
| 225 | "$id": "https://hydrilla.koszko.org/package_source-1.schema.json", |
||
| 226 | "title": "Package source", |
||
| 227 | "description": "Definition of a Hydrilla source package", |
||
| 228 | "type": "object", |
||
| 229 | "required": [ |
||
| 230 | "source_schema_version", |
||
| 231 | "source_name", |
||
| 232 | "copyright", |
||
| 233 | "upstream_url" |
||
| 234 | ], |
||
| 235 | "properties": { |
||
| 236 | "source_schema_version": { |
||
| 237 | "description": "Which version of Hydrilla source package schema this definition conforms to", |
||
| 238 | "type": "array", |
||
| 239 | "minItems": 1, |
||
| 240 | "items": [{ |
||
| 241 | "const": 1 |
||
| 242 | }], |
||
| 243 | "additionalItems": { |
||
| 244 | "type": "integer", |
||
| 245 | "minimum": 0 |
||
| 246 | }, |
||
| 247 | "minItems": 1 |
||
| 248 | }, |
||
| 249 | "source_name": { |
||
| 250 | "description": "Unique identifier of this source package", |
||
| 251 | "type": "string", |
||
| 252 | "pattern": "^[-0-9a-z.]+$" |
||
| 253 | }, |
||
| 254 | "copyright": { |
||
| 255 | "description": "Which files from the source package indicate its license terms and should be included in the distribution packages", |
||
| 256 | "type": "array", |
||
| 257 | "items": { |
||
| 258 | "$ref": "#/definitions/file_ref" |
||
| 259 | } |
||
| 260 | }, |
||
| 261 | "upstream_url": { |
||
| 262 | "description": "Where this software/work initially comes from", |
||
| 263 | "type": "string" |
||
| 264 | }, |
||
| 265 | "comment": { |
||
| 266 | "description": "An optional comment", |
||
| 267 | "type": "string" |
||
| 268 | }, |
||
| 269 | "definitions": { |
||
| 270 | "description": "Definitions of site resources and pattern->payload mappings", |
||
| 271 | "type": "array", |
||
| 272 | "items": { |
||
| 273 | "allOf": [{ |
||
| 274 | "description": "Item definition (fields common to both resources and mappings)", |
||
| 275 | "type": "object", |
||
| 276 | "required": [ |
||
| 277 | "type", |
||
| 278 | "identifier", |
||
| 279 | "long_name", |
||
| 280 | "uuid", |
||
| 281 | "version", |
||
| 282 | "description" |
||
| 283 | ], |
||
| 284 | "properties": { |
||
| 285 | "type": { |
||
| 286 | "description": "What kind of item is being defined (resource or mapping)", |
||
| 287 | "enum": ["resource", "mapping"] |
||
| 288 | }, |
||
| 289 | "identifier": { |
||
| 290 | "$ref": "#/definitions/item_identifier" |
||
| 291 | }, |
||
| 292 | "long_name": { |
||
| 293 | "description": "User-friendly alternative to an identifier", |
||
| 294 | "type": "string" |
||
| 295 | }, |
||
| 296 | "uuid": { |
||
| 297 | "description": "UUIDv4 of this item (shared with other versions of this item, otherwise unique)", |
||
| 298 | "type": "string", |
||
| 299 | "pattern": "^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$" |
||
| 300 | }, |
||
| 301 | "version": { |
||
| 302 | "description": "Item's version number", |
||
| 303 | "type": "array", |
||
| 304 | "minItems": 1, |
||
| 305 | "items": { |
||
| 306 | "type": "integer", |
||
| 307 | "minimum": 0 |
||
| 308 | }, |
||
| 309 | "contains": { |
||
| 310 | "type": "integer", |
||
| 311 | "minimum": 1 |
||
| 312 | }, |
||
| 313 | "minItems": 1 |
||
| 314 | }, |
||
| 315 | "description": { |
||
| 316 | "description": "Item's description", |
||
| 317 | "type": "string" |
||
| 318 | } |
||
| 319 | } |
||
| 320 | }, { |
||
| 321 | "anyOf": [{ |
||
| 322 | "$ref": "#/definitions/resource_definition" |
||
| 323 | }, { |
||
| 324 | "$ref": "#/definitions/mapping_definition" |
||
| 325 | }] |
||
| 326 | }] |
||
| 327 | } |
||
| 328 | } |
||
| 329 | }, |
||
| 330 | "definitions": { |
||
| 331 | "file_ref": { |
||
| 332 | "type": "object", |
||
| 333 | "required": ["file"], |
||
| 334 | "properties": { |
||
| 335 | "file": { |
||
| 336 | "description": "Filename relative to source package main directory; separator is '/'", |
||
| 337 | "type": "string", |
||
| 338 | "pattern": "^[^/]" |
||
| 339 | } |
||
| 340 | } |
||
| 341 | }, |
||
| 342 | "item_identifier": { |
||
| 343 | "description": "Identifier of an item (shared with other versions of the item, otherwise unique)", |
||
| 344 | "type": "string", |
||
| 345 | "pattern": "^[-0-9a-z]+$" |
||
| 346 | }, |
||
| 347 | "resource_definition": { |
||
| 348 | "description": "Resource definition (fields specific to resource)", |
||
| 349 | "type": "object", |
||
| 350 | "required": [ |
||
| 351 | "type", |
||
| 352 | "revision" |
||
| 353 | ], |
||
| 354 | "properties": { |
||
| 355 | "type": { |
||
| 356 | "description": "Identify this item as a resource", |
||
| 357 | "const": "resource" |
||
| 358 | }, |
||
| 359 | "revision": { |
||
| 360 | "description": "Which revision of a packaging of given version of an upstream resource is this", |
||
| 361 | "type": "integer", |
||
| 362 | "minimum": 1 |
||
| 363 | }, |
||
| 364 | "scripts": { |
||
| 365 | "description": "Which files from the source package are scripts to be included in the resource", |
||
| 366 | "type": "array", |
||
| 367 | "items":{ |
||
| 368 | "$ref": "#/definitions/file_ref" |
||
| 369 | }, |
||
| 370 | "default": [] |
||
| 371 | }, |
||
| 372 | "dependencies": { |
||
| 373 | "description": "What other resources this one depends on", |
||
| 374 | "type": "array", |
||
| 375 | "items": { |
||
| 376 | "$ref": "#/definitions/item_identifier" |
||
| 377 | }, |
||
| 378 | "default": [] |
||
| 379 | } |
||
| 380 | } |
||
| 381 | }, |
||
| 382 | "mapping_definition": { |
||
| 383 | "description": "Mapping definition (fields specific to mapping)", |
||
| 384 | "type": "object", |
||
| 385 | "required": ["type"], |
||
| 386 | "properties": { |
||
| 387 | "type": { |
||
| 388 | "description": "Identify this item as a mapping", |
||
| 389 | "const": "mapping" |
||
| 390 | }, |
||
| 391 | "payloads": { |
||
| 392 | "description": "Which payloads are to be applied to which URLs", |
||
| 393 | "additionalProperties": { |
||
| 394 | "description": "Which payload applies to URLs matching given pattern", |
||
| 395 | "type": "object", |
||
| 396 | "required": ["identifier"], |
||
| 397 | "properties": { |
||
| 398 | "identifier": { |
||
| 399 | "$ref": "#/definitions/item_identifier" |
||
| 400 | } |
||
| 401 | } |
||
| 402 | }, |
||
| 403 | "default": {}, |
||
| 404 | "examples": [{ |
||
| 405 | "https://hydrillabugs.koszko.org/***": { |
||
| 406 | "identifier": "helloapple" |
||
| 407 | }, |
||
| 408 | "https://*.koszko.org/***": { |
||
| 409 | "identifier": "hello-potato" |
||
| 410 | } |
||
| 411 | }] |
||
| 412 | } |
||
| 413 | } |
||
| 414 | } |
||
| 415 | } |
||
| 416 | } |
||
| 417 | |||
| 418 | 1 | koszko | ``` |