Real life AI: Converting Webcams into Basic Weather Stations

Leveraging Webcam Image Classification to Construct Virtual Weather Stations

Numerical weather prediction (NWP) models such as the Global Forecast System (GFS) or the European Center for Medium-Range Weather Forecasts’ (ECMWF) system are largely dependent on input validation data for accuracy. These models interpret snapshots of the current atmosphere, applying physical calculations to predict future weather conditions. Although these NWPs also rely on aviation and satellite data from various atmospheric layers, we’ll be focusing primarily on ground validation data that is less than 10 meters in height.

The precision and intricacy of input validation data directly correlate to the increased accuracy of the models’ output.

Developed regions with a dense network of weather stations, as anticipated, enjoy more accurate forecasts due to the abundance of ground observations. As shown in the following figure, the worldwide SYNOP (surface synoptic observations) weather station distribution reveals glaring gaps in certain regions.

Automated and manned weather stations make SYNOP weather observations every 6 hours to provide input for most numerical weather models, which are calculated 4-times daily.

For higher temporal resolution, METAR weather observation reports, typically generated every 30 minutes, are used. These reports are produced by airports and disseminated through organizations like NOAA, WMO, or DWD. However, the roughly 5433 stations worldwide leave even more noticeable gaps in certain regions.

Given the high cost and maintenance requirements of standard weather stations, as well as the risk of theft, these devices are not feasible solutions for developing countries. Embracing Google’s innovative approach with Google Maps—using existing, freely accessible data rather than installing new infrastructure—we can potentially utilize an abundant, often overlooked source of environmental information: webcams.

Can freely available webcam feeds provide valuable insights into current weather conditions?

We have a dataset encompassing four different environmental conditions:

1. sunshine (called shine in the dataset)
2. rain
3. clouds
4. sunrise The dataset, in its entirety, contains 1054 individual images. We’ll be using two common convolutional neural network (CNN) models, RESNET34 and RESNET50, both pretrained with the ImageNet dataset, which houses over 14 million images.

Here’s a single batch from our dataset (note: clouds are included twice in this batch):

Initially, we utilize RESNET34, a pretrained image classification network.

After only 10 minutes of programming, we achieve an error rate below 6%:

By implementing RESNET50, we exceed 5% error rate and achieve 96.44% accuracy.

The conclusion? Yes, we can discern weather conditions from webcam images.

For farmers, precipitation and cloud coverage are among the most vital weather parameters. Utilizing deep neural networks, we can classify these features with remarkable precision using freely accessible webcam images. The method showcased here managed to validate weather conditions like cloud coverage or rainfall with impressive accuracy (96.44%). It’s crucial to note that real-life application of the model might result in varying accuracy due to the relatively small dataset. However, the implications are substantial. While conventional weather stations measure numerous factors—temperature, rainfall volume, pressure, and more—that webcams currently cannot, our research suggests a promising alternative. By incorporating webcams as supplementary weather data sources, we can mitigate gaps in weather station coverage, thereby improving numerical weather predictions, particularly in developing countries, where this technology could prove indispensable for local farmers.

Update (December 2020): We’re excited to announce a partnership with Webcamgalore, a major free webcam provider, to elevate this approach.

Source of the Dataset:

Ajayi, Gbeminiyi (2018), “Multi-class Weather Dataset for Image Classification”, Mendeley Data, V1, doi: 10.17632/4drtyfjtfy.1

Access the code notebook here, or explore it now as a Python file.

weather_webcam_classifier.py

''' 
example to classify weather on webcam images
Marvin Gabler (c) MIT License
'''

from fastai import *
from fastai.vision import *
import  matplotlib.pyplot as plt
from PIL import Image
import os

def build_dataset():
    image_path = Path('.')/'images'
    fnames = get_image_files(image_path)
    
    np.random.seed(2)
    pat = r'/([^/0-9]+)\d+.(jpg|jpeg)$'
    
    # create dataset
    data = ImageDataBunch.from_name_re(image_path,
                                       fnames, 
                                       pat, 
                                       ds_tfms=get_transforms(), 
                                       size=244, 
                                       bs=4)
    data.normalize(imagenet_stats)
    
    return data


def main(data, save=True):
    # initialize learner (very conviniend in fastai)
    learn = cnn_learner(data,
                        models.resnet50,
                        metrics=[
                                accuracy_thresh,
                                error_rate
                                ],
                        callback_fns=[ShowGraph])
    
    # train the network
    learn.fit_one_cycle(4, 1e-2)
    
    if save:
        learn.save('weather_webcam_resnet50')
    
    # predict some images
    for i in os.listdir("images"):
        path = os.path.join("images",i)
        img = open_image(path)
        pred_class,pred_idx,outputs = learn.predict(img)
        print(i, pred_class.obj)
    
if __name__ == "__main__":
    main(build_dataset())