Extending wis2box¶

At its core, wis2box is a plugin architecture orchestrating all the required components of a node in the WIS2 network. Driven by topic hierarchies, wis2box can be used to process and publish any type of geospatial data beyond the requirements of the WIS2 itself.

In this section we will to explore how wis2box can be extended. wis2box plugin development requires knowledge of how to program in Python as well as Python’s packaging and module system.

Building your own data plugin¶

The heart of a wis2box data plugin is driven from the wis2box.data.base abstract base class (ABC) located in wis2box/data/base.py. Any wis2box plugin needs to inherit from wis2box.data.base.BaseAbstractData. A minimal example can be found below:

from datetime import datetime
from wis2box.data.base import BaseAbstractData

class MyCoolData(BaseAbstractData):
    """Observation data"""
    def __init__(self, defs: dict) -> None:
        super().__init__(defs)

    def transform(self, input_data: Path) -> bool:
        # transform data
        # populate self.output_data with a dict as per:
        self.output_data = {
            'c123': {
                '_meta': {
                    'identifier': 'c123',
                    'relative_filepath': '/path/to/item/',
                    'data_date': datetime_object
                },
                'bufr4': bytes(12356),
                'geojson': geojson_string
            }
        }
        return True

The key function that plugin needs to implement is the transform function. This function should return a True or False of the result of the processing, as well as populate the output_data property.

The output_data property is a dict of keys/values. Each key should be the identifier of the item, with the following values dict:

The _meta element can include the following:

identifier: identifier for report (WIGOS_<WSI>_<ISO8601>)
relative_filepath: path to data, required to publish data with BaseAbstractData.publish
geometry: GeoJSON geometry object, required to send geometry with WIS2 notification
md5: md5 checksum of encoded data
wigos_station_identifier: WIGOS identifier
data_date: (as Python datetime objects) based on the observed datetime
originating_centre: Originating centre (see Common code table C11)
data_category: Category of data, see BUFR Table A

<format-extension>: 1..n properties for each format representation, with the key being the filename extension. The value of this property can be a string or bytes, depending on whether the underlying data is ASCII or binary, for example

Packaging¶

The next step is assembling your plugin using standard Python packaging. All plugin code and configuration files should be made part of the package so that it can operate independently when running in wis2box. For distribution and installation, you have the following options:

publish to the Python Package Index (PyPI) and install in the wis2node container with pip3 install wis2box-mypackage
git clone or download your package, and install via python3 setup.py install

See the Python packaging tutorial or Cookiecutter PyPackage for guidance and templates/examples.

Note

It is recommended to name your wis2box packages with the convention wis2box-MYPLUGIN-NAME, as well as adding the keywords/topics wis2box and plugin to help discovery on platforms such as GitHub.

Integration¶

Once your package is installed on the wis2box-management container, the data mappings need to be updated to connect your plugin to a topic hierarchy. See Data mappings for more information.

An example plugin for proof of concept can be found in https://github.com/wmo-cop/wis2box-csv-observations

Example plugins¶

The following plugins provide useful examples of wis2box plugins implemented by downstream applications.

Plugin(s)	Organization/Project	Description
wis2box-csv-observations	WMO	plugin for CSV surface observation data
wis2box-pyopencdms-plugin	OpenCDMS	plugin for connecting the Open Climate Data Management System to wis2box

WIS2 in a box