Real life AI: Converting Webcams into Basic Weather Stations

01 Aug 2020

Can webcams become weather stations?

Numerical weather prediction models like GFS and ECMWF depend on ground validation data for accuracy. They take snapshots of the current atmosphere and apply physical calculations to predict future conditions. Better input data means better forecasts.

Regions with dense weather station networks get more accurate forecasts. The worldwide SYNOP (surface synoptic observations) distribution shows obvious gaps.

Automated and manned weather stations make SYNOP weather observations every 6 hours to provide input for most numerical weather models, which are calculated 4-times daily.

For higher temporal resolution, METAR weather observation reports, typically generated every 30 minutes, are used. These reports are produced by airports and disseminated through organizations like NOAA, WMO, or DWD. However, the roughly 5433 stations worldwide leave even more noticeable gaps in certain regions.

Standard weather stations are expensive to install and maintain, and in developing countries they get stolen. The alternative: use data that already exists. The same idea behind Google Maps. There are freely accessible webcams everywhere, and they contain weather information.

The experiment

We have a dataset with four weather conditions:

1. sunshine (called shine in the dataset)
2. rain
3. clouds
4. sunrise 1054 images total. We use two pretrained CNNs: RESNET34 and RESNET50, both pretrained on ImageNet (14 million+ images).

Here’s a single batch from our dataset (note: clouds are included twice in this batch):

Starting with RESNET34:

Below 6% error rate after 10 minutes of programming:

RESNET50: 96.44% accuracy.

Yes, webcams can tell you the weather.

Precipitation and cloud coverage are the parameters that matter most for farmers. With a pretrained CNN and freely accessible webcam images, we classify these at 96.44% accuracy. The dataset is small, so real-world accuracy will vary. And webcams obviously can’t measure temperature, pressure, or rainfall volume the way a proper station can.

But as supplementary data sources, they can fill gaps in weather station coverage and improve forecasts, especially in developing countries where conventional stations are not an option.

Update (December 2020): We partnered with Webcamgalore, a major free webcam provider, to take this further.

Source of the Dataset:

Ajayi, Gbeminiyi (2018), “Multi-class Weather Dataset for Image Classification”, Mendeley Data, V1, doi: 10.17632/4drtyfjtfy.1

Access the code notebook here, or explore it now as a Python file.

weather_webcam_classifier.py

''' 
example to classify weather on webcam images
Marvin Gabler (c) MIT License
'''

from fastai import *
from fastai.vision import *
import  matplotlib.pyplot as plt
from PIL import Image
import os

def build_dataset():
    image_path = Path('.')/'images'
    fnames = get_image_files(image_path)
    
    np.random.seed(2)
    pat = r'/([^/0-9]+)\d+.(jpg|jpeg)$'
    
    # create dataset
    data = ImageDataBunch.from_name_re(image_path,
                                       fnames, 
                                       pat, 
                                       ds_tfms=get_transforms(), 
                                       size=244, 
                                       bs=4)
    data.normalize(imagenet_stats)
    
    return data


def main(data, save=True):
    # initialize learner (very conviniend in fastai)
    learn = cnn_learner(data,
                        models.resnet50,
                        metrics=[
                                accuracy_thresh,
                                error_rate
                                ],
                        callback_fns=[ShowGraph])
    
    # train the network
    learn.fit_one_cycle(4, 1e-2)
    
    if save:
        learn.save('weather_webcam_resnet50')
    
    # predict some images
    for i in os.listdir("images"):
        path = os.path.join("images",i)
        img = open_image(path)
        pred_class,pred_idx,outputs = learn.predict(img)
        print(i, pred_class.obj)
    
if __name__ == "__main__":
    main(build_dataset())