IMD Radar Data#
The India Meteorological Department (IMD) provides radar data in a format similar to IRIS/Sigmet, but with some differences in structure and formatting. radarx offers specialized support for processing and analyzing this IMD radar data, ensuring compatibility and ease of use despite the variations from standard IRIS/Sigmet formats.
[1]:
import os
import subprocess
import radarx as rx
Load IMD Radar Data#
[2]:
# Base URL for the radar data files on GitHub
base_url = (
"https://raw.githubusercontent.com/syedhamidali/pyscancf_examples/main/data/goa16/"
)
# List of radar data files to download
files = [
"GOA210516024101-IMD-B.nc",
"GOA210516024101-IMD-B.nc.1",
"GOA210516024101-IMD-B.nc.2",
"GOA210516024101-IMD-B.nc.3",
"GOA210516024101-IMD-B.nc.4",
"GOA210516024101-IMD-B.nc.5",
"GOA210516024101-IMD-B.nc.6",
"GOA210516024101-IMD-B.nc.7",
"GOA210516024101-IMD-B.nc.8",
"GOA210516024101-IMD-B.nc.9",
]
# Target directory to save downloaded files
target_dir = os.path.join(os.getenv("GITHUB_WORKSPACE", "."), "radarx_data")
os.makedirs(target_dir, exist_ok=True)
# Function to download files using curl
def download_with_curl(file_name):
file_url = base_url + file_name
file_path = os.path.join(target_dir, file_name)
if not os.path.exists(file_path):
print(f"Downloading {file_name}...")
subprocess.run(["curl", "-o", file_path, file_url], check=True)
print(f"Downloaded {file_name}")
else:
print(f"{file_name} already exists.")
return file_path
# Download the files
downloaded_files = []
for file_name in files:
file_path = download_with_curl(file_name)
downloaded_files.append(file_path)
Downloading GOA210516024101-IMD-B.nc...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1413k 100 1413k 0 0 13.4M 0 --:--:-- --:--:-- --:--:-- 13.2M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
Downloaded GOA210516024101-IMD-B.nc
Downloading GOA210516024101-IMD-B.nc.1...
Downloaded GOA210516024101-IMD-B.nc.1
Downloading GOA210516024101-IMD-B.nc.2...
100 1413k 100 1413k 0 0 15.3M 0 --:--:-- --:--:-- --:--:-- 15.3M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1413k 100 1413k 0 0 12.0M 0 --:--:-- --:--:-- --:--:-- 12.0M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
Downloaded GOA210516024101-IMD-B.nc.2
Downloading GOA210516024101-IMD-B.nc.3...
Downloaded GOA210516024101-IMD-B.nc.3
Downloading GOA210516024101-IMD-B.nc.4...
100 1413k 100 1413k 0 0 11.3M 0 --:--:-- --:--:-- --:--:-- 11.3M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1413k 100 1413k 0 0 15.8M 0 --:--:-- --:--:-- --:--:-- 15.8M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1413k 100 1413k 0 0 14.0M 0 --:--:-- --:--:-- --:--:-- 14.0M
Downloaded GOA210516024101-IMD-B.nc.4
Downloading GOA210516024101-IMD-B.nc.5...
Downloaded GOA210516024101-IMD-B.nc.5
Downloading GOA210516024101-IMD-B.nc.6...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1413k 100 1413k 0 0 13.5M 0 --:--:-- --:--:-- --:--:-- 13.5M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
Downloaded GOA210516024101-IMD-B.nc.6
Downloading GOA210516024101-IMD-B.nc.7...
Downloaded GOA210516024101-IMD-B.nc.7
Downloading GOA210516024101-IMD-B.nc.8...
100 1413k 100 1413k 0 0 15.0M 0 --:--:-- --:--:-- --:--:-- 15.0M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1413k 100 1413k 0 0 12.7M 0 --:--:-- --:--:-- --:--:-- 12.7M
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
Downloaded GOA210516024101-IMD-B.nc.8
Downloading GOA210516024101-IMD-B.nc.9...
Downloaded GOA210516024101-IMD-B.nc.9
100 1413k 100 1413k 0 0 7639k 0 --:--:-- --:--:-- --:--:-- 7639k
[3]:
print(f"Number of files: {len(downloaded_files)}")
Number of files: 10
Read a Sweep#
[4]:
# Load the first radar file using radarx
swp = rx.io.read_sweep(downloaded_files[0])
swp
[4]:
<xarray.Dataset> Size: 6MB
Dimensions: (sweep: 1, time: 360, range: 996)
Coordinates:
azimuth (time) float32 1kB ...
elevation (time) float32 1kB ...
* time (time) datetime64[ns] 3kB 2021-05-16T02:41:04 ... ...
* range (range) float64 8kB 1e+03 1.25e+03 ... 2.498e+05
Dimensions without coordinates: sweep
Data variables: (12/55)
time_coverage_start |S20 20B b'2021-05-16T02:41:01Z'
fixed_angle (sweep) float32 4B 0.1978
latitude float32 4B ...
longitude float32 4B ...
altitude float32 4B ...
firstGateRange float32 4B 1e+03
... ...
sweep_mode (sweep) |S20 20B b'azimuth_surveillance'
scan_type (sweep) |S3 3B b'ppi'
time_coverage_end |S20 20B b'2021-05-16T02:41:32Z'
sweep_number (sweep) int64 8B 0
sweep_start_ray_index (sweep) int64 8B 0
sweep_end_ray_index (sweep) int64 8B 359
Attributes:
Conventions: CF/Radial instrument_parameters
version: 1.3
title:
institution: India Meteorological Department
references:
source:
comment: im/radarx
instrument_name:
history:
field_names: DBT, DBZ, VEL, WIDTHWe have successfully read one sweep in the above cell. Now, let’s create a volume scan.
Create a Volume#
We have a total of 10 files, each representing a different sweep. Our goal is to combine these individual sweeps into a single CF-Radial volume scan dataset.
[5]:
for file in downloaded_files:
print(os.path.basename(file))
GOA210516024101-IMD-B.nc
GOA210516024101-IMD-B.nc.1
GOA210516024101-IMD-B.nc.2
GOA210516024101-IMD-B.nc.3
GOA210516024101-IMD-B.nc.4
GOA210516024101-IMD-B.nc.5
GOA210516024101-IMD-B.nc.6
GOA210516024101-IMD-B.nc.7
GOA210516024101-IMD-B.nc.8
GOA210516024101-IMD-B.nc.9
[6]:
vol = rx.io.read_volume(downloaded_files)
INFO:radarx.io.imd:Successfully grouped files into 1 sweep groups.
[7]:
vol
[7]:
<xarray.DatasetView> Size: 0B
Dimensions: ()
Data variables:
*empty*We have successfully generated a volume, which can now be explored further. The volume is stored in a DataTree, a high-level structure that organizes and manages the data within the xarray framework.
[8]:
vol.groups
[8]:
('/', '/volume_0')
[9]:
ds = vol["volume_0"].ds
[10]:
ds
[10]:
<xarray.DatasetView> Size: 57MB
Dimensions: (time: 3600, range: 996, sweep: 10)
Coordinates:
azimuth (time) float32 14kB 360.0 0.9943 2.01 ... 358.0 359.0
elevation (time) float32 14kB 0.6372 0.6372 ... 21.4 21.4
* time (time) datetime64[ns] 29kB 2021-05-16T02:41:04 ......
* range (range) float64 8kB 1e+03 1.25e+03 ... 2.498e+05
Dimensions without coordinates: sweep
Data variables: (12/55)
time_coverage_start |S20 20B b'2021-05-16T02:41:01Z'
latitude float32 4B ...
longitude float32 4B ...
altitude float32 4B ...
firstGateRange float32 4B 1e+03
gateSize float32 4B 250.0
... ...
ray_angle_res (sweep) float32 40B 1.0 1.0 1.0 1.0 ... 1.0 1.0 1.0
sweep_mode (sweep) |S20 200B b'azimuth_surveillance' ... b'az...
scan_type (sweep) |S3 30B b'ppi' b'ppi' ... b'ppi' b'ppi'
sweep_number (sweep) int64 80B 0 1 2 3 4 5 6 7 8 9
sweep_start_ray_index (sweep) int64 80B 0 360 720 1080 ... 2520 2880 3240
sweep_end_ray_index (sweep) int64 80B 359 719 1079 ... 2879 3239 3599
Attributes:
Conventions: CF/Radial instrument_parameters
version: 1.3
title:
institution: India Meteorological Department
references:
source:
comment: im/radarx
instrument_name:
history:
field_names: DBT, DBZ, VEL, WIDTHWe have successfully created a volume, and we can export it to CF-Radial formatted NetCDF using xarray.
[11]:
# We can save all these cfradial1 files by iterating over the vol object
outdir = "IMD_OUT"
os.makedirs(outdir, exist_ok=True)
for key in vol.children:
ds = vol[key].ds
time_str = ds.time.min().dt.strftime("%Y%m%d_%H%M%S").item()
ds.to_netcdf(os.path.join(outdir, f"GOA_{time_str}_cfrad1.nc"))
[12]:
os.listdir("IMD_OUT/")
[12]:
['GOA_20210516_024102_cfrad1.nc']
Convert to CF/Radial2#
We can also convert it to cfradial2 data using to_cfradial2 function
[13]:
dtree = rx.io.to_cfradial2_volumes(vol)
[14]:
dtree.groups
[14]:
('/',
'/volume_0',
'/volume_0/radar_parameters',
'/volume_0/georeferencing_correction',
'/volume_0/radar_calibration',
'/volume_0/sweep_0',
'/volume_0/sweep_1',
'/volume_0/sweep_2',
'/volume_0/sweep_3',
'/volume_0/sweep_4',
'/volume_0/sweep_5',
'/volume_0/sweep_6',
'/volume_0/sweep_7',
'/volume_0/sweep_8',
'/volume_0/sweep_9')
[15]:
ds = dtree["volume_0"].ds
[16]:
# Export all the data in a dtree
# for volume in dtree:
# dtree[volume].to_netcdf()