Skip to content
BranchesTags

Repository files navigation

Hyperlocal Weather Forecasting System for Jetson Orin NX

A complete, production-ready weather forecasting system for the Seeed Studio reComputer J4012 (NVIDIA Jetson Orin NX 16GB). This system provides:

  • 18-hour HRRR forecasts from NOAA (High-Resolution Rapid Refresh model)
  • 2-hour radar nowcasting using PySTEPS and NEXRAD data
  • Web dashboard for viewing forecasts and radar
  • Automated data pipeline with continuous operation
  • Optimized for edge deployment on Jetson hardware

Features

🌡️ Hyperlocal HRRR Forecasting

  • Hourly ingestion of NOAA HRRR GRIB2 data
  • Point forecasts for Franklin, TN (configurable for any location)
  • 18-hour forecasts with temperature, wind, precipitation, cloud cover
  • Historical forecast storage for accuracy verification

📡 Radar Nowcasting

  • Real-time NEXRAD Level 2/3 radar from Nashville (KOHX)
  • PySTEPS-based ensemble nowcasting (1-2 hours)
  • Fallback to optical flow or persistence methods
  • GPU-accelerated inference with TensorRT optimization
  • Animated forecast loops

🖥️ Web Dashboard

  • Real-time current conditions
  • Interactive 18-hour forecast charts
  • Radar nowcast visualization
  • System statistics and monitoring
  • Mobile-responsive design

⚙️ Production-Ready

  • Automated data pipeline with error handling
  • systemd service integration
  • Docker containerization option
  • Resource limits for Jetson memory constraints
  • Logging and monitoring
  • Automatic data cleanup

Hardware Requirements

  • Device: Seeed Studio reComputer J4012 (Jetson Orin NX 16GB)
  • OS: Ubuntu 20.04/22.04 (L4T)
  • JetPack: 5.x (tested with 5.1.2)
  • CUDA: 11.4+
  • cuDNN: 8.6+
  • TensorRT: 8.5+
  • Storage: 64GB+ recommended (for data retention)
  • Network: Reliable internet connection for data downloads

Quick Start

1. Clone the Repository

cd ~
git clone <your-repo-url>
cd weather

2. Configure Your Location

Edit config/config.yaml:

location:
  name: "Your City, State"
  latitude: 35.9251    # Your latitude
  longitude: -86.8689  # Your longitude (negative for West)
  radar_site: "KOHX"   # Nearest NEXRAD site

Find your nearest NEXRAD site: https://www.ncei.noaa.gov/maps/nexrad/

3. Run Installation Script

chmod +x scripts/install.sh
./scripts/install.sh

The installer will:

  • Install all system dependencies
  • Set up Python packages
  • Initialize the database
  • Configure deployment (systemd or Docker)

Installation takes 15-30 minutes on Jetson.

4. Access the Dashboard

Open your browser to: http://your-jetson-ip:8080

The dashboard will show data once the first ingestion cycle completes (within 1 hour).

Installation Details

System Dependencies

The following packages are installed automatically:

  • Python 3.8+ development tools
  • HDF5, NetCDF, GRIB libraries (for weather data)
  • GEOS, PROJ (geospatial processing)
  • FFTW, OpenBLAS (numerical computing)
  • OpenCV (image processing)
  • SQLite (database)

Python Packages

Key dependencies (see requirements.txt for complete list):

  • herbie-data: HRRR GRIB2 data download
  • nexradaws: NEXRAD radar data from AWS
  • pyart: Radar data processing
  • pysteps: Precipitation nowcasting
  • xarray, dask: Scientific data handling
  • Flask: Web framework
  • SQLAlchemy: Database ORM

JetPack 5.x Compatibility

This system is designed for JetPack 5.x. Package versions are carefully selected to ensure ARM compatibility and avoid dependency conflicts.

Important: Some packages may need to compile from source on ARM, which is why installation takes longer than on x86_64.

Deployment Options

Option 1: systemd Services (Recommended)

Automated services that run on boot:

# View service status
sudo systemctl status weather-web
sudo systemctl status weather-hrrr.timer
sudo systemctl status weather-radar.timer
sudo systemctl status weather-nowcast.timer

# View logs
sudo journalctl -u weather-hrrr -f
sudo journalctl -u weather-nowcast -f

# Restart services
sudo systemctl restart weather-web

Service Schedule:

  • HRRR ingestion: Every hour at :05
  • Radar ingestion: Every 5 minutes
  • Nowcast generation: Every 10 minutes
  • Data cleanup: Daily at 2 AM
  • Web dashboard: Always running

Option 2: Docker Containers

cd deployment/docker

# Build images
docker-compose build

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f weather-web

# Stop services
docker-compose down

Note: Requires NVIDIA Container Toolkit for GPU access.

Option 3: Manual Operation

# Run full pipeline once
python3 scripts/run_pipeline.py

# Run individual components
python3 scripts/run_pipeline.py --hrrr-only
python3 scripts/run_pipeline.py --radar-only
python3 scripts/run_pipeline.py --nowcast-only

# Start web server (production-safe)
gunicorn --bind 0.0.0.0:8080 --factory src.web.app:create_app

Project Structure

weather/
├── config/
│   └── config.yaml           # Main configuration
├── src/
│   ├── ingestion/
│   │   ├── hrrr_ingestor.py      # HRRR data download
│   │   └── nexrad_ingestor.py    # NEXRAD radar download
│   ├── forecasting/
│   │   └── nowcast_engine.py     # PySTEPS nowcasting
│   ├── database/
│   │   ├── models.py             # Database schema
│   │   └── db_manager.py         # Database operations
│   ├── web/
│   │   ├── app.py                # Flask application
│   │   ├── templates/            # HTML templates
│   │   └── static/               # CSS, JavaScript
│   └── utils/
│       ├── logger.py             # Logging utilities
│       └── helpers.py            # Helper functions
├── scripts/
│   ├── run_pipeline.py       # Main pipeline orchestrator
│   └── install.sh            # Installation script
├── deployment/
│   ├── systemd/              # systemd service files
│   └── docker/               # Docker configuration
├── data/                     # Data storage (created at runtime)
├── logs/                     # Log files (created at runtime)
├── requirements.txt          # Python dependencies
└── README.md                 # This file

Configuration

Edit config/config.yaml to customize:

Location Settings

location:
  name: "Franklin, TN"
  latitude: 35.9251
  longitude: -86.8689
  timezone: "America/Chicago"
  radar_site: "KOHX"

Data Sources

hrrr:
  enabled: true
  forecast_hours: 18
  retention_days: 7

nexrad:
  enabled: true
  data_source: "aws"  # or "noaa"
  retention_hours: 24

Nowcast Settings

nowcast:
  enabled: true
  method: "pysteps"  # or "persistence", "optical_flow"
  forecast_minutes: 120
  ensemble_members: 3  # Reduce if memory constrained

Performance Tuning

performance:
  max_memory_mb: 12288      # 12GB (leave 4GB for system)
  cpu_threads: 6            # Jetson Orin NX has 8 cores
  gpu_memory_fraction: 0.7

Memory Management

The Jetson Orin NX has 16GB of unified memory. This system is configured to stay within safe limits:

  • HRRR ingestion: ~1-2GB
  • Radar ingestion: ~1-2GB
  • Nowcasting: ~4-6GB (peak during PySTEPS)
  • Web dashboard: ~500MB
  • System overhead: ~2-3GB

Tips for Memory-Constrained Operation:

  1. Reduce ensemble members in nowcast:

    nowcast:
  ensemble_members: 1

  2. Use simpler nowcast method:

    nowcast:
  method: "optical_flow"  # Uses less memory than PySTEPS

  3. Reduce forecast hours:

    hrrr:
  forecast_hours: 12

Troubleshooting

Issue: Installation fails with compilation errors

Solution: Some packages need to compile on ARM. Ensure build tools are installed:

sudo apt-get install build-essential python3-dev gfortran

Issue: HRRR download fails

Possible causes:

  • Network connectivity issues
  • NOAA server downtime
  • Incorrect herbie-data installation

Check logs:

tail -f logs/hrrr_ingestor.log

Issue: Radar data not available

Possible causes:

  • AWS S3 access issues
  • Incorrect radar site code
  • NEXRAD maintenance

Verify radar site: Check that your radar site is correct in config/config.yaml

Issue: Nowcast fails with memory error

Solution: Reduce resource usage:

nowcast:
  ensemble_members: 1
  method: "persistence"

Issue: Web dashboard shows "No data available"

Check:

  1. Are services running? sudo systemctl status weather-hrrr.timer
  2. Check logs: sudo journalctl -u weather-hrrr -n 100
  3. Verify database: sqlite3 data/weather.db "SELECT COUNT(*) FROM hrrr_forecasts;"

Issue: High CPU/memory usage

Monitor with:

# Install jtop (Jetson stats)
sudo pip3 install jetson-stats
sudo jtop

Reduce load:

  • Increase timer intervals in systemd service files
  • Reduce concurrent downloads in config.yaml

Data Sources

HRRR (High-Resolution Rapid Refresh)

  • Provider: NOAA/NCEP
  • Resolution: 3 km
  • Update: Hourly
  • Forecast: 18 hours
  • Access: Via herbie-data (AWS S3 mirror)

NEXRAD (Next Generation Radar)

  • Provider: NOAA/NWS
  • Sites: 159 across USA
  • Update: ~5 minutes
  • Access: AWS S3 (NOAA Big Data Program)

Performance

Typical execution times on Jetson Orin NX:

  • HRRR ingestion (18 hours): 2-5 minutes
  • Radar download: 10-30 seconds
  • PySTEPS nowcast (2 hours): 1-3 minutes
  • Optical flow nowcast: 10-20 seconds
  • Database cleanup: < 10 seconds

API Endpoints

The web dashboard exposes these REST API endpoints:

  • GET /api/current - Current conditions
  • GET /api/forecast - 18-hour HRRR forecast
  • GET /api/nowcast - 2-hour radar nowcast
  • GET /api/stats - System statistics
  • GET /health - Health check

Example:

curl http://localhost:8080/api/current | jq .

Development

Running Tests

# Install test dependencies
pip3 install pytest pytest-cov

# Run tests
pytest tests/

Adding a New Location

  1. Edit config/config.yaml
  2. Find coordinates: https://www.latlong.net/
  3. Find nearest NEXRAD: https://www.ncei.noaa.gov/maps/nexrad/
  4. Restart services

Customizing the Dashboard

Edit files in src/web/:

  • templates/index.html - HTML structure
  • static/css/style.css - Styling
  • static/js/app.js - Frontend logic

Contributing

Contributions welcome! Areas for improvement:

  • Additional nowcast methods (ML-based)
  • Bias correction using local observations
  • Multi-location support
  • Mobile app
  • Alert notifications
  • Verification metrics dashboard

License

MIT License - See LICENSE file for details

Acknowledgments

  • NOAA for HRRR and NEXRAD data
  • PySTEPS team for nowcasting library
  • Herbie developers for GRIB2 access
  • NVIDIA for Jetson platform

Support

For issues and questions:

  • Create a GitHub Issue
  • Check the troubleshooting section above
  • Review logs in logs/ directory

Built for edge computing and hyperlocal weather forecasting

About

No description, website, or topics provided.

Resources

Readme
MIT license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages