Using Python, requests and Pandas

Python is a popular programming language which is heavily used in the data science domains. Python provides high level functionality supporting rapid application development with a large ecosystem of packages to work with weather/climate/water data.

Let’s use the Python requests package to further interact with the wis2box API, and Pandas to run some simple summary statistics.

import json

import requests

def pretty_print(input):
    print(json.dumps(input, indent=2))


# define the endpoint of the OGC API
api = 'http://localhost:8999/pygeoapi'

Stations

Let’s find all the stations in our wis2box:

url = f'{api}/collections/stations/items?limit=50'

response = requests.get(url).json()

print(f"Number of stations: {response['numberMatched']}")

print('Stations:\n')
for station in response['features']:
    print(station['properties']['name'])

Number of stations: 19
Stations:

BALAKA
BILIRA
CHIDOOLE
CHIKANGAWA
CHIKWEO
CHINGALE
KASIYA AWS
KASUNGU NATIONAL PARK AWS
KAWALAZI
KAYEREKERA
LENGWE NATIONAL PARK
LOBI AWS
MAKANJIRA
MALOMO
MLOMBA
MTOSA BENGA
NAMITAMBO
NKHOMA UNIVERSITY
TOLEZA

Discovery Metadata

Now, let’s find all the dataset that are provided by the above stations. Each dataset is identified by a WIS 2.0 discovery metadata record.

url = f'{api}/collections/discovery-metadata/items'

response = requests.get(url).json()

print('Datasets:\n')
for dataset in response['features']:
    print(f"id: {dataset['properties']['id']}, title: {dataset['properties']['title']}")

Datasets:

id: data.core.observations-surface-land.mw.FWCL.landFixed, title: Surface weather observations (hourly)

Let’s find all the data access links associated with the Surface weather observations (hourly) dataset:

dataset_id = 'data.core.observations-surface-land.mw.FWCL.landFixed'

url = f"{api}/collections/discovery-metadata/items/{dataset_id}"

response = requests.get(url).json()

print('Data access links:\n')
for link in response['associations']:
    print(f"{link['href']} ({link['type']})")

[link['href'] for link in response['associations']]

Data access links:

http://localhost:8999/pygeoapi/collections/data.core.observations-surface-land.mw.FWCL.landFixed (OAFeat)
mqtt://mosquitto/ (MQTT)

['http://localhost:8999/pygeoapi/collections/data.core.observations-surface-land.mw.FWCL.landFixed',
 'mqtt://mosquitto/']

Let’s use the OGC API - Features (OAFeat) link to drill into the observations for Chidoole station

dataset_api_link = [link['href'] for link in response['associations'] if link['type'] == 'OAFeat'][0]

dataset_api_link

'http://localhost:8999/pygeoapi/collections/data.core.observations-surface-land.mw.FWCL.landFixed'

Observations

Let’s inspect some of the data in the API’s raw GeoJSON format:

url = f'{dataset_api_link}/items'

query_parameters = {
    'wigos_station_identifier': '0-454-2-AWSCHIDOOLE',
    'limit': 10000
}

response = requests.get(url, params=query_parameters).json()

pretty_print(response['features'][0])

{
  "id": "WIGOS_0-454-2-AWSCHIDOOLE_20220119T125500",
  "conformsTo": [
    "http://www.opengis.net/spec/ogcapi-features-1/1.0/req/geojson",
    "http://www.wmo.int/spec/om-profile-1/-/req/geojson"
  ],
  "type": "Feature",
  "geometry": {
    "type": "Point",
    "coordinates": [
      34.5,
      -15.47,
      929.0
    ]
  },
  "properties": {
    "identifier": "WIGOS_0-454-2-AWSCHIDOOLE_20220119T125500",
    "phenomenonTime": "2022-01-19T12:55:00+00:00",
    "resultTime": "2022-02-21T15:27:56+00:00",
    "wigos_station_identifier": "0-454-2-AWSCHIDOOLE",
    "metadata": [
      {
        "name": "height_of_station_above_ground_level",
        "value": 929.0,
        "units": "m"
      }
    ],
    "observations": {
      "air_pressure": {
        "value": 90903.14,
        "units": "Pa",
        "metadata": [
          {
            "name": "sensor_height_above_mean_sea_level",
            "value": 930.0,
            "units": "m"
          }
        ]
      },
      "pressure_at_mean_sea_level": {
        "value": 101623.7,
        "units": "Pa",
        "metadata": [
          {
            "name": "sensor_height_above_mean_sea_level",
            "value": 930.0,
            "units": "m"
          }
        ]
      },
      "change_of_air_pressure_over_past_3_hours": {
        "value": null,
        "units": "Pa",
        "metadata": [
          {
            "name": "sensor_height_above_mean_sea_level",
            "value": 930.0,
            "units": "m"
          }
        ]
      },
      "characteristic_of_pressure_tendency": {
        "value": 4.0,
        "units": "CODE TABLE",
        "metadata": [
          {
            "name": "sensor_height_above_mean_sea_level",
            "value": 930.0,
            "units": "m"
          }
        ]
      },
      "air_temperature": {
        "value": 24.25,
        "units": "Celsius",
        "metadata": [
          {
            "name": "sensor_height_above_local_ground",
            "value": 1.5,
            "units": "m"
          }
        ]
      },
      "dew_point_temperature": {
        "value": 21.25,
        "units": "Celsius",
        "metadata": [
          {
            "name": "sensor_height_above_local_ground",
            "value": 1.5,
            "units": "m"
          }
        ]
      },
      "relative_humidity": {
        "value": 83.0,
        "units": "%",
        "metadata": [
          {
            "name": "sensor_height_above_local_ground",
            "value": 1.5,
            "units": "m"
          }
        ]
      },
      "duration_of_sunshine_1hr": {
        "value": 0.0,
        "units": "min",
        "metadata": [
          {
            "name": "time_period",
            "value": -1.0,
            "units": "h"
          }
        ]
      },
      "duration_of_sunshine_24h": {
        "value": 0.0,
        "units": "min",
        "metadata": [
          {
            "name": "time_period",
            "value": -24.0,
            "units": "h"
          }
        ]
      },
      "precipitation_amount_1h": {
        "value": 0.0,
        "units": "kg m-2",
        "metadata": [
          {
            "name": "time_period",
            "value": -1.0,
            "units": "h"
          },
          {
            "name": "sensor_height_above_local_ground",
            "value": 1.5,
            "units": "m"
          }
        ]
      },
      "air_temperature_maximum": {
        "value": 24.55000000000001,
        "units": "Celsius",
        "metadata": [
          {
            "name": "cell_methods",
            "description": "maximum"
          },
          {
            "name": "time_period_start",
            "value": -24.0,
            "units": "h"
          },
          {
            "name": "time_period_end",
            "value": 0.0,
            "units": "h"
          },
          {
            "name": "sensor_height_above_local_ground",
            "value": 1.5,
            "units": "m"
          }
        ]
      },
      "air_temperature_minimum": {
        "value": 23.650000000000034,
        "units": "Celsius",
        "metadata": [
          {
            "name": "cell_methods",
            "description": "minimum"
          },
          {
            "name": "time_period_start",
            "value": -24.0,
            "units": "h"
          },
          {
            "name": "time_period_end",
            "value": 0.0,
            "units": "h"
          },
          {
            "name": "sensor_height_above_local_ground",
            "value": 1.5,
            "units": "m"
          }
        ]
      },
      "wind_from_direction": {
        "value": 104.0,
        "units": "deg",
        "metadata": [
          {
            "name": "cell_methods",
            "value": 2.0,
            "units": "CODE TABLE"
          },
          {
            "name": "time_period",
            "value": -10.0,
            "units": "min"
          },
          {
            "name": "sensor_height_above_local_ground",
            "value": 2.0,
            "units": "m"
          },
          {
            "name": "wind_sensor_type",
            "value": 0.0,
            "units": "FLAG TABLE"
          }
        ]
      },
      "wind_speed": {
        "value": 0.878,
        "units": "m/s",
        "metadata": [
          {
            "name": "cell_methods",
            "value": 2.0,
            "units": "CODE TABLE"
          },
          {
            "name": "time_period",
            "value": -10.0,
            "units": "min"
          },
          {
            "name": "sensor_height_above_local_ground",
            "value": 2.0,
            "units": "m"
          },
          {
            "name": "wind_sensor_type",
            "value": 0.0,
            "units": "FLAG TABLE"
          }
        ]
      },
      "wind_speed_maximum_gust": {
        "value": 2.64,
        "units": "m/s",
        "metadata": [
          {
            "name": "cell_methods",
            "value": null,
            "units": "CODE TABLE"
          },
          {
            "name": "time_period",
            "value": null,
            "units": "min"
          },
          {
            "name": "sensor_height_above_local_ground",
            "value": 2.0,
            "units": "m"
          },
          {
            "name": "wind_sensor_type",
            "value": 0.0,
            "units": "FLAG TABLE"
          }
        ]
      },
      "surface_downwelling_shortwave_flux_in_air_1h": {
        "value": 287336.3,
        "units": "J m-2",
        "metadata": [
          {
            "name": "cell_methods",
            "description": "sum"
          },
          {
            "name": "time_period",
            "value": -1.0,
            "units": "h"
          }
        ]
      },
      "surface_downwelling_shortwave_flux_in_air_24h": {
        "value": 287336.3,
        "units": "J m-2",
        "metadata": [
          {
            "name": "cell_methods",
            "description": "sum"
          },
          {
            "name": "time_period",
            "value": -24.0,
            "units": "h"
          }
        ]
      }
    },
    "id": "WIGOS_0-454-2-AWSCHIDOOLE_20220119T125500"
  }
}

Let’s inspect what’s measured at Chidoole:

print('Observed properties:\n')
for key, value in response['features'][0]['properties']['observations'].items():
    print(f'{key} ({value["units"]})')

Observed properties:

air_pressure (Pa)
pressure_at_mean_sea_level (Pa)
change_of_air_pressure_over_past_3_hours (Pa)
characteristic_of_pressure_tendency (CODE TABLE)
air_temperature (Celsius)
dew_point_temperature (Celsius)
relative_humidity (%)
duration_of_sunshine_1hr (min)
duration_of_sunshine_24h (min)
precipitation_amount_1h (kg m-2)
air_temperature_maximum (Celsius)
air_temperature_minimum (Celsius)
wind_from_direction (deg)
wind_speed (m/s)
wind_speed_maximum_gust (m/s)
surface_downwelling_shortwave_flux_in_air_1h (J m-2)
surface_downwelling_shortwave_flux_in_air_24h (J m-2)

Pandas

Let’s use the GeoJSON to build a more user-friendly table

import pandas as pd

datestamp = [obs['properties']['phenomenonTime'] for obs in response['features']]
air_temperature = [obs['properties']['observations']['air_temperature']['value'] for obs in response['features']]

d = {
    'Date/Time': datestamp,
    'Air temperature (°C)': air_temperature
}

df = pd.DataFrame(data=d)

df

	Date/Time	Air temperature (°C)
0	2022-01-19T12:55:00+00:00	24.25
1	2022-01-19T13:55:00+00:00	25.35
2	2022-01-19T14:55:00+00:00	24.55
3	2022-01-19T15:55:00+00:00	23.45
4	2022-01-19T16:55:00+00:00	21.95
...	...	...
151	2022-01-29T10:55:00+00:00	27.05
152	2022-01-29T11:55:00+00:00	29.95
153	2022-01-29T12:55:00+00:00	28.55
154	2022-01-29T13:55:00+00:00	27.35
155	2022-01-29T14:55:00+00:00	22.35

156 rows × 2 columns

print("Time extent\n")
print(f'Begin: {df["Date/Time"].min()}')
print(f'End: {df["Date/Time"].max()}')

print("Summary statistics:\n")
df[['Air temperature (°C)']].describe()

Time extent

Begin: 2022-01-19T12:55:00+00:00
End: 2022-01-29T14:55:00+00:00
Summary statistics:

	Air temperature (°C)
count	156.000000
mean	22.708974
std	2.764659
min	16.650000
25%	20.725000
50%	22.250000
75%	25.075000
max	29.950000