Compare commits

...

37 Commits

Author SHA1 Message Date
Admar Schoonen a369dabdfa Oops! Fixed the build 2022-02-18 20:15:48 +01:00
Admar Schoonen fd0725be5c Extend positions with getters to retrieve lat/lon in radians 2022-02-18 20:09:34 +01:00
Admar Schoonen 5cd2b56176 It ain't fugly if it works, right? 2022-02-18 19:51:26 +01:00
Paul van Tilburg f1a303edc0
Use address instead of position for map debug handler 2022-02-17 22:25:13 +01:00
Paul van Tilburg 4920ab4abd
Drop Weerplaza precipitation maps (closes: #8) 2022-02-17 21:47:01 +01:00
Paul van Tilburg f67f3dfe82
Only update the cache if retrieval yielded maps
* Add `is_*_stale` methods to the `MapRefresh` trait
* Only update the maps of a type if `retrieve_image` yielded something
  or if the maps are stale
* Also only then bump the timestamp for the map type

This means if there is nothing in the cache, it will retry each refresh
to get something because the timestamp is not bumped until there is
something. Once there are maps, it will only update it and bump the
timestamp if there is an image, that or, it has become stale and it
can be set to `None` and we end up in the initial state.
2022-02-17 21:38:41 +01:00
Paul van Tilburg 8d2717b392
Provide not the first map but an instant-relative map
This calculates which offset to use in the maps series with respect to
the instant of caching. It assumes the first map is current for the
instant it was retrieved.

* Rename `*_first` to `*_at` methods
* For convenience, change the types of `*_MAP_COUNT` to `u32`
* Introduce `*_MAP_INTERVAL` constants to indicate the number of
  seconds each map in the series applies to
* Return `None` if the provided instant is too far in the future
2022-02-17 21:38:16 +01:00
Paul van Tilburg 7061842bd3
Sort Cargo.toml 2022-02-16 22:20:20 +01:00
Paul van Tilburg 88b24a83ff
Move blocking image load to separate task 2022-02-16 22:02:32 +01:00
Paul van Tilburg f4a12dacdb Add a debug handler for marking a position on a map
The map that is used depends on the selected metric.
2022-02-15 17:06:32 +01:00
Paul van Tilburg 9531114eec Add methods to get the first map of each type
Also introduce constants for the number of maps included in a single
`DynamicImage`.
2022-02-15 17:04:04 +01:00
Paul van Tilburg 3a48f234e9 Introduce the Position struct; add position module
* Use `Position` everywhere instead of latitude/longitude float values
* Implement `Partial`, `Eq` and `Hash` for `Position` so it can
  part of a cache key
* Drop the `cache_key` helper function
* Rename the `address_position` function to `resolve_address`
* Add methods on `Position` for formatting latitude/longitude with
  a given precision (used for URL parameters in providers)
2022-02-15 14:15:59 +01:00
Paul van Tilburg b2f63db6b4 Mention open issues in documentation; tweak docs 2022-02-15 14:15:21 +01:00
Paul van Tilburg c76e2315b5 Split off forecast stuff to a separate module 2022-02-15 13:14:01 +01:00
Paul van Tilburg 0c5367f87f
Document caching; increase Luchtmeetnet caching to 30min 2022-02-14 21:40:07 +01:00
Paul van Tilburg 79981314d3
Make output messages more consistent 2022-02-14 21:13:35 +01:00
Paul van Tilburg 8d19dbb517
Implement caching for provider get requests (closes: #2)
* Also cache address geocoding requests to OSM Nomatim!
* Use the `cached` crate for an easy implementation
* Add the `cache_key` helper function to deal with floats being annoying
* Cache Buienradar get request for 5 minutes (per position/metric)
* Cache Luchtmeetnet get request for 5 minutes (per position/metric)
* Note the `Item` structs need to implement `Clone` now because
  the cache will own them and Rocket will want a copy too
2022-02-14 21:06:31 +01:00
Paul van Tilburg 927cb0ad92
Fix geocoded address having latitude/longitude swapped 2022-02-14 21:04:31 +01:00
Paul van Tilburg c231447ce9
Be pedantic with constant number notation 😉 2022-02-13 21:31:12 +01:00
Paul van Tilburg 859288a329
Increase the refresh intervals; improve documentation 2022-02-13 21:24:26 +01:00
Paul van Tilburg 309c79d83c
Disable retrieving precipitation maps for now (see #8) 2022-02-13 21:23:48 +01:00
Paul van Tilburg 576bcc6640
Fix UV index maps base URL
The typo was introduced in commit d432bb4.
2022-02-13 16:55:21 +01:00
Paul van Tilburg f6b26c9659
Fix expect message 2022-02-13 16:55:03 +01:00
Paul van Tilburg 79dac18655
Parse Buienradar as CSV file using serde
* Add the `csv` crate as a dependency
* Use the `Row` struct as intermediate object
* Turn the `parse_value` function into the `convert_value` function that
  cannot fail
2022-02-13 16:46:02 +01:00
Paul van Tilburg 4232263a45
Hook up the Buienradar provider metric in the forecast
Also, sync up the Luchtmeetnet provider documentation a bit.
2022-02-13 15:39:09 +01:00
Paul van Tilburg 6279d379ab
Add the Buienradar provider 2022-02-13 15:38:17 +01:00
Paul van Tilburg d432bb4cd6
Use URL objects instead of formatted strings
Use `request::Url` for this, so we don't have to depend on the `url`
crate ourselves.

Also, make the URL constants more uniform.
2022-02-13 13:10:12 +01:00
Paul van Tilburg 66abc9c4db
Hook up the Luchtmeetnet provider metrics in the forecast 2022-02-13 12:46:42 +01:00
Paul van Tilburg 59c177d508
Add the Luchtmeetnet provider
Also introduce the providers module.
2022-02-13 12:45:27 +01:00
Paul van Tilburg cbd686bd60
Small documentation improvements 2022-02-13 11:22:22 +01:00
Paul van Tilburg cf77dbb5e7
Refactor maps cache to hold lock as short as possible
This makes the response time way more snappy when the maps thread
is updating its cache.

* Move the `MapsHandle` type to the `maps` module
* SWitch to using the standard library mutex
* Split refresh methods into retrieval methods that don't need the lock
  and check timestamp & update methods that only need it shortly
* Introduce the `MapsRefresh` trait and implement it for `MapsHandle`
* Reorder some methods for clarity
* Small documentation fixes
2022-02-13 11:22:02 +01:00
Paul van Tilburg 9b9b1a5f77
Refactor so that no static is necessary for the maps cache
* Replace the lazy `once_cell` by a maps handle type
* Use Rocket's managed state to manage a handle
* Ensure that the handlers have access to it
* Pass another handle to the maps updater loop
* Try to keep the lock as short as possible

Still, long downloads block the lock. Add a FIXME to refactor this
so the lock is only taken when updating the maps fields.
2022-02-12 21:35:58 +01:00
Paul van Tilburg 72fe9577bd
Implement retrieving and caching maps 2022-02-12 21:08:13 +01:00
Paul van Tilburg d058ab4448
Set up a global maps (cache) object 2022-02-12 17:20:36 +01:00
Paul van Tilburg b5dae45868
Create maps cache and run its task next to Rocket 2022-02-12 17:12:06 +01:00
Paul van Tilburg 6b24c4f6e7
Move blocking geocding forward resolving to a separate thread 2022-02-12 17:12:01 +01:00
Paul van Tilburg ae2d2c1c56 Merge pull request 'Clarify that PAQI is the combination of pollen and air quality index' (#1) from admar/sinoptik:clarify-PAQI into main
Reviewed-on: paul/sinoptik#1
2022-02-12 17:10:45 +01:00
9 changed files with 1645 additions and 152 deletions

740
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -4,6 +4,12 @@ version = "0.1.0"
edition = "2021"
[dependencies]
cached = { version = "0.30.0", features = ["async"] }
chrono = "0.4.19"
chrono-tz = "0.6.1"
color-eyre = "0.5.6"
csv = "1.1.6"
geocoding = "0.3.1"
image = "0.24.0"
reqwest = { version = "0.11.9", features = ["json"] }
rocket = { version = "0.5.0-rc.1", features = ["json"] }

148
src/forecast.rs Normal file
View File

@ -0,0 +1,148 @@
//! Forecast retrieval and construction.
//!
//! This module is used to construct a [`Forecast`] for the given position by retrieving data for
//! the requested metrics from their providers.
use rocket::serde::Serialize;
use crate::maps::MapsHandle;
use crate::position::Position;
use crate::providers;
use crate::providers::buienradar::Item as BuienradarItem;
use crate::providers::luchtmeetnet::Item as LuchtmeetnetItem;
/// The current forecast for a specific location.
///
/// Only the metrics asked for are included as well as the position and current time.
///
// TODO: Fill in missing data (#3)
#[derive(Debug, Default, Serialize)]
#[serde(crate = "rocket::serde")]
pub(crate) struct Forecast {
/// The latitude of the position.
lat: f64,
/// The longitude of the position.
lon: f64,
/// The current time (in seconds since the UNIX epoch).
time: i64,
/// The air quality index (when asked for).
#[serde(rename = "AQI", skip_serializing_if = "Option::is_none")]
aqi: Option<Vec<LuchtmeetnetItem>>,
/// The NO₂ concentration (when asked for).
#[serde(rename = "NO2", skip_serializing_if = "Option::is_none")]
no2: Option<Vec<LuchtmeetnetItem>>,
/// The O₃ concentration (when asked for).
#[serde(rename = "O3", skip_serializing_if = "Option::is_none")]
o3: Option<Vec<LuchtmeetnetItem>>,
/// The combination of pollen + air quality index (when asked for).
#[serde(rename = "PAQI", skip_serializing_if = "Option::is_none")]
paqi: Option<()>,
/// The particulate matter in the air (when asked for).
#[serde(rename = "PM10", skip_serializing_if = "Option::is_none")]
pm10: Option<Vec<LuchtmeetnetItem>>,
/// The pollen in the air (when asked for).
#[serde(skip_serializing_if = "Option::is_none")]
pollen: Option<()>,
/// The precipitation (when asked for).
#[serde(skip_serializing_if = "Option::is_none")]
precipitation: Option<Vec<BuienradarItem>>,
/// The UV index (when asked for).
#[serde(rename = "UVI", skip_serializing_if = "Option::is_none")]
uvi: Option<()>,
}
impl Forecast {
fn new(position: Position) -> Self {
Self {
lat: position.lat,
lon: position.lon,
time: chrono::Utc::now().timestamp(),
..Default::default()
}
}
}
/// The supported forecast metrics.
///
/// This is used for selecting which metrics should be calculated & returned.
#[allow(clippy::upper_case_acronyms)]
#[derive(Copy, Clone, Debug, Eq, Hash, PartialEq, rocket::FromFormField)]
pub(crate) enum Metric {
/// All metrics.
#[field(value = "all")]
All,
/// The air quality index.
AQI,
/// The NO₂ concentration.
NO2,
/// The O₃ concentration.
O3,
/// The combination of pollen + air quality index.
PAQI,
/// The particulate matter in the air.
PM10,
/// The pollen in the air.
Pollen,
/// The precipitation.
Precipitation,
/// The UV index.
UVI,
}
impl Metric {
/// Returns all supported metrics.
fn all() -> Vec<Metric> {
use Metric::*;
Vec::from([AQI, NO2, O3, PAQI, PM10, Pollen, Precipitation, UVI])
}
}
/// Calculates and returns the forecast.
///
/// The provided list `metrics` determines what will be included in the forecast.
pub(crate) async fn forecast(
position: Position,
metrics: Vec<Metric>,
_maps_handle: &MapsHandle,
) -> Forecast {
let mut forecast = Forecast::new(position);
// Expand the `All` metric if present, deduplicate otherwise.
let mut metrics = metrics;
if metrics.contains(&Metric::All) {
metrics = Metric::all();
} else {
metrics.dedup()
}
for metric in metrics {
match metric {
// This should have been expanded to all the metrics matched below.
Metric::All => unreachable!("The all metric should have been expanded"),
Metric::AQI => forecast.aqi = providers::luchtmeetnet::get(position, metric).await,
Metric::NO2 => forecast.no2 = providers::luchtmeetnet::get(position, metric).await,
Metric::O3 => forecast.o3 = providers::luchtmeetnet::get(position, metric).await,
Metric::PAQI => forecast.paqi = Some(()),
Metric::PM10 => forecast.pm10 = providers::luchtmeetnet::get(position, metric).await,
Metric::Pollen => forecast.pollen = Some(()),
Metric::Precipitation => {
forecast.precipitation = providers::buienradar::get(position, metric).await
}
Metric::UVI => forecast.uvi = Some(()),
}
}
forecast
}

View File

@ -11,170 +11,207 @@
)]
#![deny(missing_docs)]
use geocoding::{Forward, Openstreetmap, Point};
use std::sync::{Arc, Mutex};
use color_eyre::Result;
use rocket::http::ContentType;
use rocket::response::content::Custom;
use rocket::serde::json::Json;
use rocket::serde::Serialize;
use rocket::{get, launch, routes, FromFormField};
use rocket::tokio::time::Instant;
use rocket::tokio::{self, select};
use rocket::{get, routes, State};
/// The current for a specific location.
///
/// Only the metrics asked for are included as well as the position and current time.
///
/// TODO: Fill the metrics with actual data!
#[derive(Debug, Default, PartialEq, Serialize)]
#[serde(crate = "rocket::serde")]
struct Forecast {
/// The latitude of the position.
lat: f64,
use std::f64::consts::PI;
/// The longitude of the position.
lon: f64,
pub(crate) use self::forecast::Metric;
use self::forecast::{forecast, Forecast};
pub(crate) use self::maps::{Maps, MapsHandle};
use self::position::{resolve_address, Position};
/// The current time (in seconds since the UNIX epoch).
time: i64,
/// The air quality index (when asked for).
#[serde(rename = "AQI", skip_serializing_if = "Option::is_none")]
aqi: Option<u8>,
/// The NO₂ concentration (when asked for).
#[serde(rename = "NO2", skip_serializing_if = "Option::is_none")]
no2: Option<u8>,
/// The O₃ concentration (when asked for).
#[serde(rename = "O3", skip_serializing_if = "Option::is_none")]
o3: Option<u8>,
/// The combination of pollen + air quality index (when asked for).
#[serde(rename = "PAQI", skip_serializing_if = "Option::is_none")]
paqi: Option<u8>,
/// The particulate matter in the air (when asked for).
#[serde(rename = "PM10", skip_serializing_if = "Option::is_none")]
pm10: Option<u8>,
/// The pollen in the air (when asked for).
#[serde(skip_serializing_if = "Option::is_none")]
pollen: Option<u8>,
/// The precipitation (when asked for).
#[serde(skip_serializing_if = "Option::is_none")]
precipitation: Option<u8>,
/// The UV index (when asked for).
#[serde(rename = "UVI", skip_serializing_if = "Option::is_none")]
uvi: Option<u8>,
}
impl Forecast {
fn new(lat: f64, lon: f64) -> Self {
let time = chrono::Utc::now().timestamp();
Self {
lat,
lon,
time,
..Default::default()
}
}
}
/// The supported metrics.
///
/// This is used for selecting which metrics should be calculated & returned.
#[allow(clippy::upper_case_acronyms)]
#[derive(Copy, Clone, Debug, Eq, PartialEq, FromFormField)]
enum Metric {
/// All metrics.
#[field(value = "all")]
All,
/// The air quality index.
AQI,
/// The NO₂ concentration.
NO2,
/// The O₃ concentration.
O3,
/// The combination of pollen + air quality index.
PAQI,
/// The particulate matter in the air.
PM10,
/// The pollen in the air.
Pollen,
/// The precipitation.
Precipitation,
/// The UV index.
UVI,
}
impl Metric {
/// Returns all supported metrics.
fn all() -> Vec<Metric> {
use Metric::*;
Vec::from([AQI, NO2, O3, PAQI, PM10, Pollen, Precipitation, UVI])
}
}
/// Calculates and returns the forecast.
///
/// The provided list `metrics` determines what will be included in the forecast.
async fn forecast(lat: f64, lon: f64, metrics: Vec<Metric>) -> Forecast {
let mut forecast = Forecast::new(lat, lon);
// Expand the `All` metric if present, deduplicate otherwise.
let mut metrics = metrics;
if metrics.contains(&Metric::All) {
metrics = Metric::all();
} else {
metrics.dedup()
}
for metric in metrics {
match metric {
// This should have been expanded to all the metrics matched below.
Metric::All => unreachable!("should have been expanded"),
Metric::AQI => forecast.aqi = Some(1),
Metric::NO2 => forecast.no2 = Some(2),
Metric::O3 => forecast.o3 = Some(3),
Metric::PAQI => forecast.paqi = Some(4),
Metric::PM10 => forecast.pm10 = Some(5),
Metric::Pollen => forecast.pollen = Some(6),
Metric::Precipitation => forecast.precipitation = Some(7),
Metric::UVI => forecast.uvi = Some(8),
}
}
forecast
}
/// Retrieves the geocoded position for the given address.
async fn address_position(address: &str) -> Option<(f64, f64)> {
let osm = Openstreetmap::new();
// FIXME: Handle or log the error.
let points: Vec<Point<f64>> = osm.forward(address).ok()?;
points.get(0).map(|point| (point.x(), point.y()))
}
pub(crate) mod forecast;
pub(crate) mod maps;
pub(crate) mod position;
pub(crate) mod providers;
/// Handler for retrieving the forecast for an address.
#[get("/forecast?<address>&<metrics>")]
async fn forecast_address(address: String, metrics: Vec<Metric>) -> Option<Json<Forecast>> {
let (lat, lon) = address_position(&address).await?;
let forecast = forecast(lat, lon, metrics).await;
async fn forecast_address(
address: String,
metrics: Vec<Metric>,
maps_handle: &State<MapsHandle>,
) -> Option<Json<Forecast>> {
let position = resolve_address(address).await?;
let forecast = forecast(position, metrics, maps_handle).await;
Some(Json(forecast))
}
/// Handler for retrieving the forecast for a geocoded position.
#[get("/forecast?<lat>&<lon>&<metrics>", rank = 2)]
async fn forecast_geo(lat: f64, lon: f64, metrics: Vec<Metric>) -> Json<Forecast> {
let forecast = forecast(lat, lon, metrics).await;
async fn forecast_geo(
lat: f64,
lon: f64,
metrics: Vec<Metric>,
maps_handle: &State<MapsHandle>,
) -> Json<Forecast> {
let position = Position::new(lat, lon);
let forecast = forecast(position, metrics, maps_handle).await;
Json(forecast)
}
/// Launches rocket.
#[launch]
async fn rocket() -> _ {
rocket::build().mount("/", routes![forecast_address, forecast_geo])
fn deg2rad(x: f64)->f64 {
let y: f64 = x * PI / 180.0;
return y;
}
/// Mercator projection from https://stackoverflow.com/questions/18838915/convert-lat-lon-to-pixel-coordinate
fn mercator_y(lat: f64)->f64 {
return f64::ln(f64::tan(lat / 2.0 + PI / 4.0));
}
/// Handler for showing the current map with the geocoded position for a specific metric.
///
/// Note: This handler is mosly used for debugging purposes!
#[get("/map?<address>&<metric>")]
async fn show_map(
address: String,
metric: Metric,
maps_handle: &State<MapsHandle>,
) -> Option<Custom<Vec<u8>>> {
use image::{GenericImage, Rgba};
use std::io::Cursor;
let position = resolve_address(address).await?;
let lat = position.lat;
let lon = position.lon;
let now = Instant::now();
let maps = maps_handle.lock().expect("Maps handle lock was poisoned");
let mut image = match metric {
Metric::PAQI => maps.pollen_at(now)?,
Metric::Pollen => maps.pollen_at(now)?,
Metric::UVI => maps.uvi_at(now)?,
_ => return None, // Unsupported metric
};
// GPS coordinates from Google Maps
let vlissingen_lat: f64 = deg2rad(51.44);
let vlissingen_lon: f64 = deg2rad(3.57);
let vlissingen_x: u32 = 84;
let vlissingen_y: u32 = 745;
let winschoten_lat: f64 = deg2rad(53.14);
let winschoten_lon: f64 = deg2rad(7.04);
let winschoten_x: u32 = 729;
let winschoten_y: u32 = 185;
let lauwersoog_lat: f64 = deg2rad(53.40);
let lauwersoog_lon: f64 = deg2rad(6.22);
let lauwersoog_x: u32 = 566;
let lauwersoog_y: u32 = 111;
let enschede_lat: f64 = deg2rad(52.22);
let enschede_lon: f64 = deg2rad(6.90);
let enschede_x: u32 = 694;
let enschede_y: u32 = 494;
let ref_x1 = vlissingen_x;
let ref_lon1 = vlissingen_lon;
let ref_y1 = vlissingen_y;
let ref_lat1 = vlissingen_lat;
let ref_x2 = enschede_x;
let ref_lon2 = enschede_lon;
let ref_y2 = lauwersoog_y;
let ref_lat2 = lauwersoog_lat;
let y_min = mercator_y(ref_lat1);
let y_max = mercator_y(ref_lat2);
let x_factor = ((ref_x2 - ref_x1) as f64) / (ref_lon2 - ref_lon1);
let y_factor = ((ref_y1 - ref_y2) as f64) / (y_max - y_min);
println!("x_factor: {}, y_factor: {}", x_factor, y_factor);
println!("y_min: {}, y_max: {}, lat: {}", y_min * y_factor, y_max * y_factor, mercator_y(deg2rad(lat)) * y_factor);
let mut x_f64 = (deg2rad(lon) - ref_lon1) * x_factor + ref_x1 as f64;
let mut y_f64 = (y_max - mercator_y(deg2rad(lat))) * y_factor + ref_y2 as f64;
println!("x: {}, y: {}", x_f64, y_f64);
if x_f64 < 0.0 {
x_f64 = 0.0;
}
if x_f64 > (image.width() - 1) as f64 {
x_f64 = (image.width() - 1) as f64;
}
if y_f64 < 0.0 {
y_f64 = 0.0;
}
if y_f64 > (image.height() - 1) as f64 {
y_f64 = (image.height() - 1) as f64;
}
println!("bounded to x: {}, y: {}", x_f64, y_f64);
for px in ref_x1 - 5..=ref_x1 + 5 {
for py in ref_y1 - 5..=ref_y1 + 5 {
image.put_pixel(px, py, Rgba::from([0x00, 0xff, 0xff, 0x70]));
}
}
for px in ref_x2 - 5..=ref_x2 + 5 {
for py in ref_y2 - 5..=ref_y2 + 5 {
image.put_pixel(px, py, Rgba::from([0xff, 0x00, 0xff, 0x70]));
}
}
for py in 0..(image.height() - 1) {
image.put_pixel(x_f64 as u32, py, Rgba::from([0x00, 0x00, 0x00, 0x70]));
}
for px in 0..(image.width() - 1) {
image.put_pixel(px, y_f64 as u32, Rgba::from([0x00, 0x00, 0x00, 0x70]));
}
// Encode the image as PNG image data.
// FIXME: This encoding call blocks the worker thread!
let mut image_data = Cursor::new(Vec::new());
image
.write_to(
&mut image_data,
image::ImageOutputFormat::from(image::ImageFormat::Png),
)
.ok()?;
Some(Custom(ContentType::PNG, image_data.into_inner()))
}
/// Starts the main maps refresh loop and sets up and launches Rocket.
///
/// See [`maps::run`] for the maps refresh loop.
#[rocket::main]
async fn main() -> Result<()> {
color_eyre::install()?;
let maps = Maps::new();
let maps_handle = Arc::new(Mutex::new(maps));
let maps_updater = tokio::spawn(maps::run(Arc::clone(&maps_handle)));
let rocket = rocket::build()
.manage(maps_handle)
.mount("/", routes![forecast_address, forecast_geo, show_map])
.ignite()
.await?;
let shutdown = rocket.shutdown();
select! {
result = rocket.launch() => {
result?
}
result = maps_updater => {
shutdown.notify();
result?
}
}
Ok(())
}

249
src/maps.rs Normal file
View File

@ -0,0 +1,249 @@
//! Maps retrieval and caching.
//!
//! This module provides a task that keeps maps up-to-date using a maps-specific refresh interval.
//! It stores all the maps as [`DynamicImage`]s in memory.
use std::sync::{Arc, Mutex};
use image::{DynamicImage, ImageFormat};
use reqwest::Url;
use rocket::tokio;
use rocket::tokio::time::{sleep, Duration, Instant};
/// A handle to access the in-memory cached maps.
pub(crate) type MapsHandle = Arc<Mutex<Maps>>;
/// The interval between map refreshes (in seconds).
const REFRESH_INTERVAL: Duration = Duration::from_secs(60);
/// The base URL for retrieving the pollen maps from Buienradar.
const POLLEN_BASE_URL: &str =
"https://image.buienradar.nl/2.0/image/sprite/WeatherMapPollenRadarHourlyNL\
?width=820&height=988&extension=png&renderBackground=False&renderBranding=False\
&renderText=False&history=0&forecast=24&skip=0";
/// The interval for retrieving pollen maps.
///
/// The endpoint provides a map for every hour, 24 in total.
const POLLEN_INTERVAL: Duration = Duration::from_secs(3_600);
/// The number of pollen maps retained.
const POLLEN_MAP_COUNT: u32 = 24;
/// The number of seconds each pollen map is for.
const POLLEN_MAP_INTERVAL: u64 = 3_600;
/// The base URL for retrieving the UV index maps from Buienradar.
const UVI_BASE_URL: &str = "https://image.buienradar.nl/2.0/image/sprite/WeatherMapUVIndexNL\
?width=820&height=988&extension=png&&renderBackground=False&renderBranding=False\
&renderText=False&history=0&forecast=5&skip=0";
/// The interval for retrieving UV index maps.
///
/// The endpoint provides a map for every day, 5 in total.
const UVI_INTERVAL: Duration = Duration::from_secs(24 * 3_600);
/// The number of UV index maps retained.
const UVI_MAP_COUNT: u32 = 5;
/// The number of seconds each UV index map is for.
const UVI_MAP_INTERVAL: u64 = 24 * 3_600;
/// The `MapsRefresh` trait is used to reduce the time a lock needs to be held when updating maps.
///
/// When refreshing maps, the lock only needs to be held when checking whether a refresh is
/// necessary and when the new maps have been retrieved and can be updated.
trait MapsRefresh {
/// Determines whether the pollen maps need to be refreshed.
fn needs_pollen_refresh(&self) -> bool;
/// Determines whether the UV index maps need to be refreshed.
fn needs_uvi_refresh(&self) -> bool;
/// Determines whether the pollen maps are stale.
fn is_pollen_stale(&self) -> bool;
/// Determines whether the UV index maps are stale.
fn is_uvi_stale(&self) -> bool;
/// Updates the pollen maps.
fn set_pollen(&self, pollen: Option<DynamicImage>);
/// Updates the UV index maps.
fn set_uvi(&self, uvi: Option<DynamicImage>);
}
/// Container type for all in-memory cached maps.
#[derive(Debug)]
pub(crate) struct Maps {
/// The pollen maps (from Buienradar).
pub(crate) pollen: Option<DynamicImage>,
/// The timestamp the pollen maps were last refreshed.
pollen_stamp: Instant,
/// The UV index maps (from Buienradar).
pub(crate) uvi: Option<DynamicImage>,
/// The timestamp the UV index maps were last refreshed.
uvi_stamp: Instant,
}
impl Maps {
/// Creates a new maps cache.
///
/// It contains an [`DynamicImage`] per maps type, if downloaded, and the timestamp of the last
/// update.
pub(crate) fn new() -> Self {
let now = Instant::now();
Self {
pollen: None,
pollen_stamp: now,
uvi: None,
uvi_stamp: now,
}
}
/// Returns the pollen map for the given instant.
///
/// This returns [`None`] if the map is not in the cache yet, or if `instant` is too far in the
/// future with respect to the cached maps.
pub(crate) fn pollen_at(&self, instant: Instant) -> Option<DynamicImage> {
let duration = instant.duration_since(self.pollen_stamp);
let offset = (duration.as_secs() / POLLEN_MAP_INTERVAL) as u32;
// Check if out of bounds.
if offset >= POLLEN_MAP_COUNT {
return None;
}
self.pollen.as_ref().map(|map| {
let width = map.width() / POLLEN_MAP_COUNT;
map.crop_imm(offset * width, 0, width, map.height())
})
}
/// Returns the UV index map for the given instant.
///
/// This returns [`None`] if the map is not in the cache yet, or if `instant` is too far in
/// the future with respect to the cached maps.
pub(crate) fn uvi_at(&self, instant: Instant) -> Option<DynamicImage> {
let duration = instant.duration_since(self.uvi_stamp);
let offset = (duration.as_secs() / UVI_MAP_INTERVAL) as u32;
// Check if out of bounds.
if offset >= UVI_MAP_COUNT {
return None;
}
self.uvi.as_ref().map(|map| {
let width = map.width() / UVI_MAP_COUNT;
map.crop_imm(offset * width, 0, width, map.height())
})
}
}
impl MapsRefresh for MapsHandle {
fn is_pollen_stale(&self) -> bool {
let maps = self.lock().expect("Maps handle mutex was poisoned");
Instant::now().duration_since(maps.pollen_stamp)
> Duration::from_secs(POLLEN_MAP_COUNT as u64 * POLLEN_MAP_INTERVAL)
}
fn is_uvi_stale(&self) -> bool {
let maps = self.lock().expect("Maps handle mutex was poisoned");
Instant::now().duration_since(maps.uvi_stamp)
> Duration::from_secs(UVI_MAP_COUNT as u64 * UVI_MAP_INTERVAL)
}
fn needs_pollen_refresh(&self) -> bool {
let maps = self.lock().expect("Maps handle mutex was poisoned");
maps.pollen.is_none() || Instant::now().duration_since(maps.pollen_stamp) > POLLEN_INTERVAL
}
fn needs_uvi_refresh(&self) -> bool {
let maps = self.lock().expect("Maps handle mutex was poisoned");
maps.uvi.is_none() || Instant::now().duration_since(maps.uvi_stamp) > UVI_INTERVAL
}
fn set_pollen(&self, pollen: Option<DynamicImage>) {
if pollen.is_some() || self.is_pollen_stale() {
let mut maps = self.lock().expect("Maps handle mutex was poisoned");
maps.pollen = pollen;
maps.pollen_stamp = Instant::now();
}
}
fn set_uvi(&self, uvi: Option<DynamicImage>) {
if uvi.is_some() || self.is_uvi_stale() {
let mut maps = self.lock().expect("Maps handle mutex was poisoned");
maps.uvi = uvi;
maps.uvi_stamp = Instant::now();
}
}
}
/// Retrieves an image from the provided URL.
///
/// This returns [`None`] if it fails in either performing the request, retrieving the bytes from
/// the image or loading and the decoding the data into [`DynamicImage`].
async fn retrieve_image(url: Url) -> Option<DynamicImage> {
// TODO: Handle or log errors!
let response = reqwest::get(url).await.ok()?;
let bytes = response.bytes().await.ok()?;
tokio::task::spawn_blocking(move || {
image::load_from_memory_with_format(&bytes, ImageFormat::Png)
})
.await
.ok()?
.ok()
}
/// Retrieves the pollen maps from Buienradar.
///
/// See [`POLLEN_BASE_URL`] for the base URL and [`retrieve_image`] for the retrieval function.
async fn retrieve_pollen_maps() -> Option<DynamicImage> {
let timestamp = format!("{}", chrono::Local::now().format("%y%m%d%H%M"));
let mut url = Url::parse(POLLEN_BASE_URL).unwrap();
url.query_pairs_mut().append_pair("timestamp", &timestamp);
println!("🔽 Refreshing pollen maps from: {}", url);
retrieve_image(url).await
}
/// Retrieves the UV index maps from Buienradar.
///
/// See [`UVI_BASE_URL`] for the base URL and [`retrieve_image`] for the retrieval function.
async fn retrieve_uvi_maps() -> Option<DynamicImage> {
let timestamp = format!("{}", chrono::Local::now().format("%y%m%d%H%M"));
let mut url = Url::parse(UVI_BASE_URL).unwrap();
url.query_pairs_mut().append_pair("timestamp", &timestamp);
println!("🔽 Refreshing UV index maps from: {}", url);
retrieve_image(url).await
}
/// Runs a loop that keeps refreshing the maps when necessary.
///
/// Use [`MapsRefresh`] trait methods on `maps_handle` to check whether each maps type needs to be
/// refreshed and uses its retrieval function to update it if necessary.
pub(crate) async fn run(maps_handle: MapsHandle) -> ! {
loop {
println!("🕔 Refreshing the maps (if necessary)...");
if maps_handle.needs_pollen_refresh() {
let pollen = retrieve_pollen_maps().await;
maps_handle.set_pollen(pollen);
}
if maps_handle.needs_uvi_refresh() {
let uvi = retrieve_uvi_maps().await;
maps_handle.set_uvi(uvi);
}
sleep(REFRESH_INTERVAL).await;
}
}

115
src/position.rs Normal file
View File

@ -0,0 +1,115 @@
//! Positions in the geographic coordinate system.
//!
//! This module contains everything related to geographic coordinate system functionality.
use std::hash::Hash;
use cached::proc_macro::cached;
use geocoding::{Forward, Openstreetmap, Point};
use rocket::tokio;
use std::f64::consts::PI;
/// A (geocoded) position.
///
/// This is used for measuring and communication positions directly on the Earth as latitude and
/// longitude.
///
/// # Position equivalence and hashing
///
/// For caching purposes we need to check equivalence between two positions. If the positions match
/// up to the 5th decimal, we consider them the same (see [`Position::lat_as_i32`] and
/// [`Position::lon_as_i32`]).
#[derive(Clone, Copy, Debug, Default)]
pub(crate) struct Position {
pub(crate) lat: f64,
pub(crate) lon: f64,
}
impl Position {
/// Creates a new (geocoded) position.
pub(crate) fn new(lat: f64, lon: f64) -> Self {
Self { lat, lon }
}
/// Returns the latitude as an integer.
///
/// This is achieved by multiplying it by `10_000` and rounding it. Thus, this gives a
/// precision of 5 decimals.
fn lat_as_i32(&self) -> i32 {
(self.lat * 10_000.0).round() as i32
}
/// Returns the longitude as an integer.
///
/// This is achieved by multiplying it by `10_000` and rounding it. Thus, this gives a
/// precision of 5 decimals.
fn lon_as_i32(&self) -> i32 {
(self.lon * 10_000.0).round() as i32
}
/// Returns the latitude as a string with the given precision.
pub(crate) fn lat_as_str(&self, precision: usize) -> String {
format!("{:.*}", precision, self.lat)
}
/// Returns the longitude as a string with the given precision.
pub(crate) fn lon_as_str(&self, precision: usize) -> String {
format!("{:.*}", precision, self.lon)
}
/// Returns the latitude in radians
fn lat_as_rad(&self) -> f64 {
self.lat * PI / 180.0
}
/// Returns the longitude in radians
fn lon_as_rad(&self) -> f64 {
self.lon * PI / 180.0
}
}
impl From<&Point<f64>> for Position {
fn from(point: &Point<f64>) -> Self {
// The `geocoding` API always returns (longitude, latitude) as (x, y).
Position::new(point.y(), point.x())
}
}
impl Hash for Position {
fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
// Floats cannot be hashed. Use the 5-decimal precision integer representation of the
// coordinates instead.
self.lat_as_i32().hash(state);
self.lon_as_i32().hash(state);
}
}
impl PartialEq for Position {
fn eq(&self, other: &Self) -> bool {
self.lat_as_i32() == other.lat_as_i32() && self.lon_as_i32() == other.lon_as_i32()
}
}
impl Eq for Position {}
/// Resolves the geocoded position for a given address.
///
/// Returns [`None`] if the address could not be geocoded or the OpenStreetMap Nomatim API could
/// not be contacted.
///
/// If the result is [`Some`], it will be cached.
/// Note that only the 100 least recently used addresses will be cached.
#[cached(size = 100)]
pub(crate) async fn resolve_address(address: String) -> Option<Position> {
println!("🌍 Geocoding the position of the address: {}", address);
tokio::task::spawn_blocking(move || {
let osm = Openstreetmap::new();
let points: Vec<Point<f64>> = osm.forward(&address).ok()?;
points.get(0).map(Position::from)
})
.await
.ok()
.flatten()
}

6
src/providers.rs Normal file
View File

@ -0,0 +1,6 @@
//! All supported metric data providers.
//!
//! Data is either provided via a direct (JSON) API or via looking up values on maps.
pub(crate) mod buienradar;
pub(crate) mod luchtmeetnet;

119
src/providers/buienradar.rs Normal file
View File

@ -0,0 +1,119 @@
//! The Buienradar data provider.
//!
//! For more information about Buienradar, see: <https://www.buienradar.nl/overbuienradar/contact>
//! and <https://www.buienradar.nl/overbuienradar/gratis-weerdata>.
use cached::proc_macro::cached;
use chrono::offset::TimeZone;
use chrono::serde::ts_seconds;
use chrono::{DateTime, Local, NaiveTime, ParseError, Utc};
use chrono_tz::Europe;
use csv::ReaderBuilder;
use reqwest::Url;
use rocket::serde::{Deserialize, Serialize};
use crate::position::Position;
use crate::Metric;
/// The base URL for the Buienradar API.
const BUIENRADAR_BASE_URL: &str = "https://gpsgadget.buienradar.nl/data/raintext";
/// A row in the precipitation text output.
///
/// This is an intermediate type used to represent rows of the output.
#[derive(Debug, Deserialize)]
#[serde(crate = "rocket::serde")]
struct Row {
/// The precipitation value in the range `0..=255`.
value: u16,
/// The time in the `HH:MM` format.
time: String,
}
/// The Buienradar API precipitation data item.
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(crate = "rocket::serde", try_from = "Row")]
pub(crate) struct Item {
/// The time(stamp) of the forecast.
#[serde(serialize_with = "ts_seconds::serialize")]
time: DateTime<Utc>,
/// The forecasted value.
///
/// Its unit is mm/h.
value: f32,
}
impl TryFrom<Row> for Item {
type Error = ParseError;
fn try_from(row: Row) -> Result<Self, Self::Error> {
let time = parse_time(&row.time)?;
let value = convert_value(row.value);
Ok(Item { time, value })
}
}
/// Parses a time string to date/time in the UTC time zone.
///
/// The provided time has the format `HH:MM` and is considered to be in the Europe/Amsterdam
/// time zone.
fn parse_time(t: &str) -> Result<DateTime<Utc>, ParseError> {
// First, get the naive time.
let ntime = NaiveTime::parse_from_str(t, "%H:%M")?;
// FIXME: This might actually be the day before when started on a machine that
// doesn't run in the Europe/Amsterdam time zone.
let ndtime = Local::today().naive_local().and_time(ntime);
// Then, interpret the naive date/time in the Europe/Amsterdam time zone and convert it to the
// UTC time zone.
let ldtime = Europe::Amsterdam.from_local_datetime(&ndtime).unwrap();
let dtime = ldtime.with_timezone(&Utc);
Ok(dtime)
}
/// Converts a precipitation value into an precipitation intensity value in mm/h.
///
/// For the conversion formula, see: <https://www.buienradar.nl/overbuienradar/gratis-weerdata>.
fn convert_value(v: u16) -> f32 {
let base: f32 = 10.0;
let value = base.powf((v as f32 - 109.0) / 32.0);
(value * 10.0).round() / 10.0
}
/// Retrieves the Buienradar forecasted precipitation items for the provided position.
///
/// It only supports the following metric:
/// * [`Metric::Precipitation`]
///
/// Returns [`None`] if retrieval or deserialization fails, or if the metric is not supported by
/// this provider.
///
/// If the result is [`Some`] it will be cached for 5 minutes for the the given position and
/// metric.
#[cached(time = 300, option = true)]
pub(crate) async fn get(position: Position, metric: Metric) -> Option<Vec<Item>> {
if metric != Metric::Precipitation {
return None;
}
let mut url = Url::parse(BUIENRADAR_BASE_URL).unwrap();
url.query_pairs_mut()
.append_pair("lat", &position.lat_as_str(2))
.append_pair("lon", &position.lon_as_str(2));
println!("▶️ Retrieving Buienradar data from: {url}");
let response = reqwest::get(url).await.ok()?;
let output = match response.error_for_status() {
Ok(res) => res.text().await.ok()?,
Err(_err) => return None,
};
let mut rdr = ReaderBuilder::new()
.has_headers(false)
.delimiter(b'|')
.from_reader(output.as_bytes());
rdr.deserialize().collect::<Result<_, _>>().ok()
}

View File

@ -0,0 +1,79 @@
//! The Luchtmeetnet open data provider.
//!
//! For more information about Luchtmeetnet, see: <https://www.luchtmeetnet.nl/contact>.
use cached::proc_macro::cached;
use chrono::serde::ts_seconds;
use chrono::{DateTime, Utc};
use reqwest::Url;
use rocket::serde::{Deserialize, Serialize};
use crate::position::Position;
use crate::Metric;
/// The base URL for the Luchtmeetnet API.
const LUCHTMEETNET_BASE_URL: &str = "https://api.luchtmeetnet.nl/open_api/concentrations";
/// The Luchtmeetnet API data container.
///
/// This is only used temporarily during deserialization.
#[derive(Debug, Deserialize)]
#[serde(crate = "rocket::serde")]
struct Container {
data: Vec<Item>,
}
/// The Luchtmeetnet API data item.
#[derive(Clone, Debug, Deserialize, Serialize)]
#[serde(crate = "rocket::serde")]
pub(crate) struct Item {
/// The time(stamp) of the forecast.
#[serde(
rename(deserialize = "timestamp_measured"),
serialize_with = "ts_seconds::serialize"
)]
time: DateTime<Utc>,
/// The forecasted value.
///
/// The unit depends on the selected [metric](Metric).
value: f32,
}
/// Retrieves the Luchtmeetnet forecasted items for the provided position and metric.
///
/// It supports the following metrics:
/// * [`Metric::AQI`]
/// * [`Metric::NO2`]
/// * [`Metric::O3`]
/// * [`Metric::PM10`]
///
/// Returns [`None`] if retrieval or deserialization fails, or if the metric is not supported by
/// this provider.
///
/// If the result is [`Some`] it will be cached for 30 minutes for the the given position and
/// metric.
#[cached(time = 1800, option = true)]
pub(crate) async fn get(position: Position, metric: Metric) -> Option<Vec<Item>> {
let formula = match metric {
Metric::AQI => "lki",
Metric::NO2 => "no2",
Metric::O3 => "o3",
Metric::PM10 => "pm10",
_ => return None, // Unsupported metric
};
let mut url = Url::parse(LUCHTMEETNET_BASE_URL).unwrap();
url.query_pairs_mut()
.append_pair("formula", formula)
.append_pair("latitude", &position.lat_as_str(5))
.append_pair("longitude", &position.lon_as_str(5));
println!("▶️ Retrieving Luchtmeetnet data from: {url}");
let response = reqwest::get(url).await.ok()?;
let root: Container = match response.error_for_status() {
Ok(res) => res.json().await.ok()?,
Err(_err) => return None,
};
Some(root.data)
}