Compare commits

...

176 Commits
main ... main

Author SHA1 Message Date
Paul van Tilburg ba1b27fd66
Bump the version to 0.2.12
Check, lint and test using Cargo / Check, lint and test (push) Failing after 9m37s Details
Release / Release Rust crate (push) Successful in 4m37s Details
Release / Release Debian package (push) Has been cancelled Details
Release / Release (push) Has been cancelled Details
2024-05-09 12:29:47 +02:00
Paul van Tilburg 94c29cad71
Update the changelog 2024-05-09 12:28:47 +02:00
Paul van Tilburg 8e5a1ef305
Bump dependency on reqwest to 0.12.4 2024-05-09 12:25:46 +02:00
Paul van Tilburg 6c39cac26e
Bump dependency on image to 0.25.1
Also only enable the `png` feature and update a deprecated output format
definition.
2024-05-09 12:24:44 +02:00
Paul van Tilburg f1ee03d96c
Bump dependency on chrono-tz to 0.9.0 2024-05-09 12:24:41 +02:00
Paul van Tilburg 98f60cba89
Bump dependency on cached to 0.51.3 2024-05-09 12:24:39 +02:00
Paul van Tilburg 45ee951601
Cargo update; fixes RUSTSEC-2024-0019 and RUSTSEC-2024-0332 2024-05-09 12:12:58 +02:00
Paul van Tilburg f4e7c82b53
Fix test; remove maps load/maps test race
Check, lint and test using Cargo / Check, lint and test (push) Successful in 4m39s Details
Introduce the Rocket core that can be used to test the API without
background processes interfering with the test setup.
2024-05-09 12:04:00 +02:00
Paul van Tilburg a29d7f3535
Fix test; reduce require accurancy for coordinates
There are currently two entries for the city of Eindhoven in the
Nomanitim geoecoding database. Reduced acccuracy still is enough to
check that the API works properly.
2024-05-09 12:03:48 +02:00
Paul van Tilburg c86f001fee
Fix clippy issues
Check, lint and test using Cargo / Check, lint and test (push) Failing after 5m23s Details
2024-05-03 14:34:46 +02:00
Paul van Tilburg 88c59cdb1f
Bump the version to 0.2.11
Check, lint and test using Cargo / Check, lint and test (push) Successful in 4m18s Details
Release / Release (push) Successful in 1m3s Details
Release / Release Rust crate (push) Successful in 3m1s Details
Release / Release Debian package (push) Successful in 5m12s Details
2024-02-27 16:07:22 +01:00
Paul van Tilburg cad766b520
Update the changelog 2024-02-27 16:07:19 +01:00
Paul van Tilburg e62699c102
Tweak/fix tests; reduce required accuracy for geocoded coordinates
Also somebody seems to have slightly moved Eindhoven.
2024-02-27 16:06:56 +01:00
Paul van Tilburg f32f67dbf4
Fix clippy issue 2024-02-27 16:00:57 +01:00
Paul van Tilburg d1e43a7aa7
Cargo update; fixes several security advisories
Fixes RUSTSEC-2024-0003 and RUSTSEC-2023-0072.
2024-02-27 15:59:44 +01:00
Paul van Tilburg c2450267e0
Bump the version to 0.2.10
Check, lint and test using Cargo / Check, lint and test (push) Successful in 5m42s Details
Release / Release (push) Successful in 1m35s Details
Release / Release Rust crate (push) Successful in 4m10s Details
Release / Release Debian package (push) Successful in 6m3s Details
2023-11-03 10:41:56 +01:00
Paul van Tilburg 087ecf00f1
Update the changelog 2023-11-03 10:40:43 +01:00
Paul van Tilburg f8ea25c516
Bump the dependency on cached to 0.46.0 2023-11-03 10:39:49 +01:00
Paul van Tilburg f830d34464
Cargo update; fixes RUSTSEC-2020-0071 and RUSTSEC-2023-0044
Fix the tests for small changes in Rocket 0.5-rc.4.
Also fix the usage of a deprecate method.
2023-11-03 10:39:47 +01:00
Paul van Tilburg ff10cc19e8
Correct Debian package file pattern
Check, lint and test using Cargo / Check, lint and test (push) Successful in 5m34s Details
2023-08-25 21:23:11 +02:00
Paul van Tilburg 1211fea46a
Bump the version to 0.2.9
Check, lint and test using Cargo / Check, lint and test (push) Successful in 6m10s Details
Release / Release (push) Successful in 1m47s Details
Release / Release Rust crate (push) Successful in 4m28s Details
Release / Release Debian package (push) Failing after 6m22s Details
2023-08-25 20:48:46 +02:00
Paul van Tilburg 182521aab7
Update the changelog 2023-08-25 20:48:17 +02:00
Paul van Tilburg dadf5d3147
Fix clippy issue
Check, lint and test using Cargo / Check, lint and test (push) Successful in 6m9s Details
2023-08-25 20:24:19 +02:00
Paul van Tilburg 4b506541f3
Build and release a Debian package in a separate job
Check, lint and test using Cargo / Check, lint and test (push) Failing after 3m34s Details
Release it to the package repository instead of attaching to the release.
Also add the relevant part of the changelog as release notes to the
release and fix some schema-related issues.
2023-08-25 20:15:39 +02:00
Paul van Tilburg 47e28a7098
Cargo update 2023-08-25 20:06:56 +02:00
Paul van Tilburg 07e0701106
Bump the version to 0.2.8
Check, lint and test using Cargo / Check, lint and test (push) Successful in 12m45s Details
Release / Release (push) Successful in 12m16s Details
Release / Release crate (push) Successful in 10m0s Details
2023-06-04 12:13:46 +02:00
Paul van Tilburg 91d5500c86
Update the changelog 2023-06-04 12:13:19 +02:00
Paul van Tilburg 9b3c11ee76
Cargo update 2023-06-04 12:12:54 +02:00
Paul van Tilburg 27e1ac726c
Bump dependency on cached to 0.44.0 2023-06-04 12:12:54 +02:00
Paul van Tilburg 3047cf74c2
No longer configure using a sparse Cargo index for crates.io
This is the default since Rust 1.70.
2023-06-04 12:02:35 +02:00
Paul van Tilburg 44474aa545
Tweak README
Check, lint and test using Cargo / Check, lint and test (push) Successful in 5m44s Details
2023-05-29 16:38:42 +02:00
Paul van Tilburg 50b0e94839
Properly attribute the PAQI metric
Check, lint and test using Cargo / Check, lint and test (push) Successful in 5m45s Details
2023-05-29 16:37:16 +02:00
Paul van Tilburg 1010311403
Don't provide the map for the PAQI metric (it is pollen only) 2023-05-29 16:36:02 +02:00
Paul van Tilburg d16699636b Merge pull request 'Print the version on lift off and add version endpoint' (#30) from 29-print-version-add-endpoint into main
Check, lint and test using Cargo / Check, lint and test (push) Successful in 5m52s Details
Reviewed-on: paul/sinoptik#30
2023-05-29 16:14:56 +02:00
Paul van Tilburg 38fb28c248 Add a /version API endpoint
Check, lint and test using Cargo / Check, lint and test (pull_request) Successful in 6m5s Details
Check, lint and test using Cargo / Check, lint and test (push) Has been cancelled Details
* Introduce the `VersionInfo` struct, build from the vergen environment
  variables
* Add the `version` handler to construct and return the version info
* Update the README
2023-05-29 15:48:36 +02:00
Paul van Tilburg 7c2b012e95 Print the version on lift off 2023-05-29 15:48:36 +02:00
Paul van Tilburg ab6001f072 Use the vergen crate to generate version information
* Add depend on the `vergen` crate (only use the `build`, `git` and
  `gitcl` features)
* Add the build script `build.rs` to setup the environment variables
  from the build system
2023-05-29 15:48:36 +02:00
Paul van Tilburg 9742331f6d
Annote the map key colors in the comments
Check, lint and test using Cargo / Check, lint and test (push) Successful in 5m42s Details
2023-05-26 20:44:24 +02:00
Paul van Tilburg 9bb9d248a8
Bump the version to 0.2.7
Check, lint and test using Cargo / Check, lint and test (push) Successful in 5m39s Details
Release / Release (push) Successful in 7m29s Details
Release / Release crate (push) Successful in 4m8s Details
2023-05-26 20:17:31 +02:00
Paul van Tilburg 37788fac1c
Update the changelog
Also add missing release dates!
2023-05-26 20:16:24 +02:00
Paul van Tilburg 112875e7ac
Use the personal Cargo token
Use this instead of the (missing) repository's secret.
2023-05-26 20:05:52 +02:00
Paul van Tilburg 1c71ca79ef
Switch back to the original Buienradar color scheme (refs: #27)
Check, lint and test using Cargo / Check, lint and test (push) Successful in 6m29s Details
This reverts commit a52313ffb7.
2023-05-26 19:43:37 +02:00
Paul van Tilburg afca20c96f
Bump the version to 0.2.6
Check, lint and test using Cargo / Check, lint and test (push) Successful in 5m35s Details
Release / Release (push) Successful in 8m17s Details
Release / Release crate (push) Failing after 4m12s Details
2023-05-24 22:17:59 +02:00
Paul van Tilburg 2d34eee49a
Update the changelog 2023-05-24 22:17:31 +02:00
Paul van Tilburg a52313ffb7
Switch to new Buienradar color scheme (closes: #27)
Check, lint and test using Cargo / Check, lint and test (push) Successful in 6m13s Details
2023-05-24 22:13:13 +02:00
Admar Schoonen f39a3a33ee
Set sampling area to 31x31 (closes: #26)
Check, lint and test using Cargo / Check, lint and test (push) Successful in 6m16s Details
2023-05-24 19:20:24 +02:00
Paul van Tilburg a59b4eefe1
Improve error description/comment 2023-05-24 19:16:04 +02:00
Paul van Tilburg 1aad3e2eb6
Nomatim seems to geocode Eindhoven differently now
Check, lint and test using Cargo / Check, lint and test (push) Successful in 5m32s Details
2023-05-22 20:50:46 +02:00
Paul van Tilburg 929508a9cc
Add a full release workflow
Check, lint and test using Cargo / Check, lint and test (push) Failing after 5m32s Details
2023-05-22 20:09:45 +02:00
Paul van Tilburg 23e4f731a0
Tweak step name 2023-05-22 20:08:36 +02:00
Paul van Tilburg d84440304a
Simplify Gitea Actions check, lint and test workflow
Check, lint and test using Cargo / Check, lint and test (push) Successful in 6m20s Details
2023-04-25 16:38:59 +02:00
Paul van Tilburg a289bd9ef0
Bump the version to 0.2.5
Check Details
Lints Details
Test Suite Details
2023-03-24 13:10:22 +01:00
Paul van Tilburg 122f98a92d
Update the changelog 2023-03-24 13:09:53 +01:00
Paul van Tilburg 1426405943
Bump dependencies on cached and chrono-tz 2023-03-24 13:05:42 +01:00
Paul van Tilburg 34be96d187
Update to Rocket 0.5.0-rc.3 2023-03-24 13:04:56 +01:00
Paul van Tilburg bc140a9d1e
Remove unnecessary debug statement
Check Details
Lints Details
Test Suite Details
2023-03-23 16:57:30 +01:00
Paul van Tilburg 39c224eb90
Cargo update; fixes RUSTSEC-2023-0018 2023-03-23 16:56:48 +01:00
Paul van Tilburg b517448fd7
Speed up workflow by using sparce Cargo index for crates.io
Check Details
Lints Details
Test Suite Details
2023-03-21 11:50:00 +01:00
Paul van Tilburg 3de66dbd41
Add Gitea Actions (CI) workflow for cargo
Check Details
Lints Details
Test Suite Details
2023-03-21 11:16:48 +01:00
Paul van Tilburg 6a04fc958f
Fix float comparison in tests 2023-03-21 11:15:32 +01:00
Paul van Tilburg c8b951ab7e
Fix clippy issue 2023-03-21 11:05:03 +01:00
Paul van Tilburg 32ec6b516c
Cargo update
This fixes build issues with the `geo-types` crate versio 0.7.7.
Also replace use of now deprecated functions/methods.
2023-01-31 14:02:46 +01:00
Paul van Tilburg a6301fa678 Fix markdownlint issues 2022-10-23 10:50:05 +02:00
Paul van Tilburg f00537d5f3
Add more lints; fix issues 2022-10-17 20:02:54 +02:00
Paul van Tilburg c8970fa3bb
Cargo update 2022-10-17 19:53:01 +02:00
Paul van Tilburg aee3409f4a Bump dependency on cached to 0.38.0
This fixes the unused `*_prime_cache` compile warnings.
2022-08-12 09:46:17 +02:00
Paul van Tilburg dbdd7bef0f Cargo update 2022-08-12 09:44:15 +02:00
Paul van Tilburg d749233b24 Merge uses 2022-08-12 09:44:08 +02:00
Paul van Tilburg abb6657212
Cargo update 2022-07-17 13:25:22 +02:00
Paul van Tilburg 8b03f2162b
Bump dependency on geocoding to 0.4.0
This finally removes the duplicate dependency tree on older versions of
crates we're already using (chrono, request, etc.).
2022-07-17 13:25:22 +02:00
Paul van Tilburg 789bb1d1ac
Update the changelog 2022-07-05 14:46:51 +02:00
Paul van Tilburg 2e999f5a78
Bump the version to 0.2.4 2022-07-05 14:42:51 +02:00
Paul van Tilburg 712b3a9acf
Check sample coordinate bounds (closes: #24) 2022-06-06 19:51:07 +02:00
Paul van Tilburg 2b23885692
Default to now if Last-Modified header missing
As a result, if the header is missing, it is no longer considered an
error.
2022-06-06 19:39:32 +02:00
Paul van Tilburg dc47c1c73c Merge pull request 'Handle errors internally and show them via the API' (#25) from handle-errors into main
Reviewed-on: paul/sinoptik#25
2022-06-06 16:49:03 +02:00
Paul van Tilburg 014ca5a151
Handle errors on the API side
* The map endpoints return an HTTP 404 error in case of unknown or
  out-of-bound locations
* The forecast endpoint with an address returns an HTTP 404 with error
  JSON in case geocoding fails
* The forecast endpoints return the errors per metric in the `errors`
  field of the forecast
* Implement `Display` for `Metric`
* Use a `BTreeMap` to have an ordered `errors` field/object
* Also log the errors to the console
* Update the tests
* Document the errors that can occur
2022-06-06 16:46:52 +02:00
Paul van Tilburg 8a2a6d769d
Log errors of the map refresher task separately 2022-06-06 15:38:38 +02:00
Paul van Tilburg 69ef08002c
Introduce error types, switch to Results everywhere
* Add dependency on the `thiserror` crate
* Add a global `Error` type, but also `maps::Error` and
  `providers::combined::MergeError` for convenience
* Add matching `Result` types that default to the respective `Error`
  type
* Refactor code to yield all kinds of error variants
* Add FIXMEs where library errors still need to be handled
* Remove documentation that explained why `None` was returned, this is
  captured in the error now
2022-06-06 15:37:54 +02:00
Paul van Tilburg 7d0cd4a822
Drop pollen and AQI max for PAQI metric
* This was introduced as per #20 but no longer deemed necessary
* Fix up some comments
* Keep the PAQI documentation in `README.md`
2022-06-05 21:47:12 +02:00
Paul van Tilburg fb8236696d
Use world map emoji! 2022-06-05 21:29:12 +02:00
Paul van Tilburg aab3b737be
Run map refresher as an ad hoc liftoff fairing
* Simplify the library `setup()` method
* Simplify launching Rocket
* Drop dependency on color-eyre
2022-06-05 21:25:56 +02:00
Paul van Tilburg 3451445de1
Update the changelog 2022-05-21 09:51:59 +02:00
Paul van Tilburg b88c7f77a0
Bump the version to 0.2.3 2022-05-21 09:49:18 +02:00
Paul van Tilburg 705ffae249
Fix typo in map key color for level 8 2022-05-21 09:47:23 +02:00
Paul van Tilburg bd2344beb6
Transform map key colors to hexadecimal format 2022-05-21 09:47:01 +02:00
Paul van Tilburg bad20b803a
Don't set unnecessary status 2022-05-10 15:47:37 +02:00
Paul van Tilburg fc4672328f
Fix missing type for tests
This was introduced by the changes in commit e204e79.
2022-05-10 15:47:08 +02:00
Paul van Tilburg 769f0745c8
Update examples with fixes of v0.2.2 taken into account 2022-05-10 15:26:58 +02:00
Paul van Tilburg 3fb899d1fd
Update the changelog 2022-05-10 15:05:57 +02:00
Paul van Tilburg f0cc54f074
Bump the version to 0.2.2 2022-05-10 14:58:21 +02:00
Paul van Tilburg ac653ef0c9
Cargo update 2022-05-10 14:57:03 +02:00
Paul van Tilburg e204e7905c
Update to Rocket 0.5-rc.2 2022-05-10 14:55:57 +02:00
Paul van Tilburg 08cdfe1e1c Merge pull request 'Fix timestamps for map samples not being correct' (#22) from 21-fix-ts-map-samples into main
Reviewed-on: paul/sinoptik#22
2022-05-10 14:26:38 +02:00
Paul van Tilburg 89395f21f6
Introduce RetrievedMaps; refactor around it
The `RetrievedMaps` struct captures the image and its metadata:
the last modification time and the base timestamp for the maps.

* No longer store the last modification time, called "stamp" before,
  separately in the `Maps` struct
* Update methods on `Maps` to use the `RetrievedMaps` structs and
  the timestamp base in particular for sampling and map marking
* Update the `MapsRefresh` implemention to use the last modification
  time
* Rename some variables from `map` to `image` in the helper functions
  for consistency
* Update tests and documentation
2022-05-10 14:19:09 +02:00
Paul van Tilburg ff9f1ac371
Parse timestamp base from filename 2022-05-10 13:21:21 +02:00
Paul van Tilburg 4a6eeab787
Fix sample/item being out of the combined series time range
For example, if there are 24 valid pollen samples and 20 valid air
quality items, the maximum pollen sample could be de 23th, but the
resulting combined series will only cover 20 items. So, it is should not
return that, but only look in the first 20 pollen samples for the
maximum sample.
2022-05-10 12:26:10 +02:00
Paul van Tilburg ab4b0bba72
Fix valid samples/items being discarded too early
A forecasted sample/item may be for example timestamped at 14:00.
For hourly forecasts, it will still be valid until 14:59:59, not
14:29:59.
2022-05-10 12:24:26 +02:00
Paul van Tilburg b61eb63cdf
Update the changelog 2022-05-08 14:04:05 +02:00
Paul van Tilburg e5e006dc00
Bump the version to 0.2.1 2022-05-08 14:02:47 +02:00
Paul van Tilburg a0c4e0da77
Don't use Option for the max sample/item return values
If the sample/item series are empty, the function already returns
`None`, so the tuple values are always `Some(_)` which makes the
`Option` type redundant.
2022-05-08 14:01:22 +02:00
Paul van Tilburg 34b63ec94d
Compact merge tests by using constructors
Add the constructors to sample/item structs for testing purposes.
2022-05-08 14:00:12 +02:00
Paul van Tilburg 5a23e83b7f
Extend tests for combined provider merge function
* Check that merging fails if the samples/items are too far apart
* Check that nothing is returned if either of the lists is empty
* Check that if either series is shifted, they are merged correctly
2022-05-08 13:48:09 +02:00
Paul van Tilburg e2d1a1d9df
Filter out old item/samples in combined provider
* Only retain samples/items that have timestamps that are at least half
  an hour ago
* Add a test to verify that the merge discards them
2022-05-08 12:53:32 +02:00
Paul van Tilburg 29b79c720d
Derive PartialEq for most item/sample structs 2022-05-08 12:53:09 +02:00
Paul van Tilburg 5d37c5b5ee
Bump the version to 0.2.0 2022-05-07 21:55:03 +02:00
Paul van Tilburg 2c04a92965
Update the changelog 2022-05-07 21:54:53 +02:00
Paul van Tilburg e408fbb91d
Add a changelog 2022-05-07 21:50:59 +02:00
Paul van Tilburg 5972697cf1
Yield pollen and AQI max for PAQI metric (closes: #20)
* Make the combined provider keep track of the AQI and pollen maximum
  value
* Extend the `Forecast` struct with the `aqi_max` and `pollen_max`
  fields
* Fill the `aqi_max` and `pollen_max` fields when the PAQI metric is
  selected
* Update the documentation
* Extend the tests
2022-05-07 21:43:35 +02:00
Paul van Tilburg 7feae97ee2
Bump dependencies; cargo update 2022-05-07 20:21:09 +02:00
Paul van Tilburg 0bf07bd134
Split off all functionality to a library crate
This way we can build Rockets from outside the crate and run benchmarks,
for example.

* Add top-level `setup()` function to create a Rocket and set up the
  maps refresher task
* Change the type of `maps::run` since `!` is still an unstable type
* Fix HTTP code blocks in `README.md` so they don't appear as doctests
  to rustdoc
2022-03-15 09:54:02 +01:00
Paul van Tilburg 4576f8d90a
More textual improvements 2022-03-11 16:35:38 +01:00
Paul van Tilburg 66bd02ea57
Also document the map API endpoint
Because, why not?
2022-03-11 16:18:04 +01:00
Paul van Tilburg 80fd9525c4
Fix missing endpoint in example URLs; tweak text a bit 2022-03-11 16:09:14 +01:00
Paul van Tilburg 32964cea21
Add basic top-level tests
This starts to address #14 but didn't turn into a full MR yet.

* Use crates `assert_float_eq` and `assert_matches` for extra assertions
* Split off a function to build a Rocket `rocket()` that can be used
  in the tests
2022-03-11 16:03:56 +01:00
Paul van Tilburg 738409c3a8
Fix bound check using constant value
It should use the actual count instead!
2022-03-07 19:19:14 +01:00
Paul van Tilburg 65a4420191
Bump dependency; cargo update
Bump the dependency on `cached` to 0.32.0 (fixes documentation)
2022-02-26 09:36:15 +01:00
Paul van Tilburg dc90f57422 Document API response; fix typo 2022-02-25 15:43:15 +01:00
Paul van Tilburg 512f170804 Bump dependencies; cargo update
* Bump the dependency on `image` to 0.24.1 (PNG bugfix)
* Bump the dependency on `cached` to 0.31.0 (I/O caching support?)
2022-02-25 15:26:44 +01:00
Paul van Tilburg 8b1dee96e0 Merge pull request 'Implement PAQI metric' (#16) from 15-implement-paqi into main
Reviewed-on: paul/sinoptik#16
2022-02-24 20:43:34 +01:00
Paul van Tilburg d33b5f1dbb
Hook up the combined metric in the forecast 2022-02-24 20:40:35 +01:00
Paul van Tilburg 95f30751c6
Add the combined provider 2022-02-24 20:40:07 +01:00
Paul van Tilburg 1ae6c896dd
Make item/sample fields available to the crate 2022-02-24 20:23:33 +01:00
Paul van Tilburg d0ace275ec
Filter out items that are older than 1h before now
Luchtmeetnet seems to yield some results that are in the past some
times?
2022-02-24 20:21:55 +01:00
Paul van Tilburg a78c55332f Refactor the API for the maps module
* Move the function `draw_position` to the maps module and split it up
* Replace and refactor `pollen_at` and `uvi_at` methods on `Maps`
  by `pollen_mark` and `uvi_mark`
* Drop the `pollen_project` and `uvi_project` methods on `Maps`,
  just call the `project` helper method directly
* Add `map_at` and `mark` helper methods that handle maps slicing
  and drawing
* Rename `pollen_sample` and `uvi_sample` methods on `Maps` to their
  plural forms
* Also, rename the map handlers to `map_address` and `map_geo`
2022-02-22 12:49:42 +01:00
Paul van Tilburg 9cd4be160c Update description; add authors and repository fields 2022-02-21 21:05:50 +01:00
Paul van Tilburg 9e9955fb58 Add Debian packaging via cargo-deb
* Add the required metadata to `Cargo.toml`
* Add a systemd unit file
* Use `Rocket.toml.example` as the default configuration
2022-02-21 20:55:00 +01:00
Paul van Tilburg 4db8e4ac03 Add Rocket.toml example file; fix syntax
Also ignore a local `Rocket.toml`; it can be used for development
without comitting it.
2022-02-21 20:54:32 +01:00
Paul van Tilburg 84a434268f
Add README.md and LICENSE file (closes: #6)
Also link the files from the crate and include `README.md` as main crate
documentation.
2022-02-21 17:48:28 +01:00
Paul van Tilburg 7432fb3cd3 Fix Buienradar timestamp madness (closes: #3)
* Use the `Europe::Amsterdam` time zone from `chrono_tz` to determine
  what date/time it is in that time zone
* Parse timestamps in the rain text API relative to this date/time
* Add the `fix_items_day_boundary` function to fix up stuff if
  the series of timestamps in the rain text cross the day boundary
2022-02-20 17:39:49 +01:00
Paul van Tilburg 17d5daeabc Use chrono (UTC) timestamps for maps (closes: #12)
* Adapt `retrieve_image` to also return a timestamp based on the
  CDN's last modified time; adapt other methods accordingly
* For the maps module, use `chrono::Utc` instead of
  `tokio::time::Instant` and use `chrono::Duration` instead of
  `tokio::time::Duration`
* Pass the maps timestamp to the `sample` function so it can use
  that timestamp as base
2022-02-19 16:45:37 +01:00
Paul van Tilburg a5ca1f02ff Merge pull request 'Implment the pollen and UV index metrics' (#13) from 4-implement-pollen-uvi into main
Reviewed-on: paul/sinoptik#13
2022-02-19 15:42:57 +01:00
Paul van Tilburg 46531a76bd Add retrieval of pollen and UVI metrics for Buienradar provider 2022-02-19 15:33:00 +01:00
Paul van Tilburg 5dc51b4c02 Implement map sampling using a map key
* Define the map key for Buienradar as `MAP_KEY` (colors used on
  Buienradar maps)
* Define a `MapkeyHistogram` type and add the `map_key_historgram()`
  function to construct one
* Define the sample size to look for pixels around the sampling
  coordinate
* Introduce a separata `sample` function that returns the samples
  for a map and the provided coordinates and starting timestamp
* Implement `Maps::pollen_sample` and Maps::uvi_sample`
2022-02-19 15:32:55 +01:00
Paul van Tilburg eb9951dbce Distinguish between a map and maps
* A map is a view into the image of concatenated maps
* Ensure that projection only happens on the first map
* Make `project` generic over all generic image views
2022-02-19 15:08:05 +01:00
Paul van Tilburg 0df30b695f Split off precipitation get code
This is necessary so that the Buienradar provider can support more
metrics using the maps.
2022-02-19 09:41:11 +01:00
Paul van Tilburg 7c24f73937 Add an extra map handler for showing lat/lon
* Split of the positioning and drawing code to the `draw_position`
  helper method
* Move the positioning and drawing code to a Tokia task
2022-02-18 23:18:50 +01:00
Paul van Tilburg a03573d20d Allow dead code for not-yet-implemented methods (for now) 2022-02-18 23:04:49 +01:00
Paul van Tilburg ac9576defa Merge pull request 'Implement Mercator projection' (#10) from 9-implement-mercator-projection into main
Reviewed-on: paul/sinoptik#10
2022-02-18 23:04:08 +01:00
Paul van Tilburg 7895b5afc9 Hook up the map projection in the map debug handler
Also change how the resulting coordinate is marked on the map
(2 line guides instead of a square).
2022-02-18 23:00:58 +01:00
Paul van Tilburg 4b80121187 Implement map projection using reference points
* Use the Mercator projection to get the image coordinates
* Fix errors in the reference points
* Tweak documentation
2022-02-18 22:59:21 +01:00
Paul van Tilburg 7d1a1a1c0d Add helper methods that calculate lat/lon in radians 2022-02-18 22:58:39 +01:00
Paul van Tilburg 6b62cc7797 Add some unimplemented API for map project and sampling
* Add map reference point constants `*_MAP_REF_POINTS `that can be used
  for map projections
* Add (unimplemented )`*_project` and `*_sample` methods to the `Maps`
  implementation
* Add `PollenSample` and `UviSample` structs
* Make `Position::new` const
2022-02-18 21:25:06 +01:00
Paul van Tilburg f1a303edc0
Use address instead of position for map debug handler 2022-02-17 22:25:13 +01:00
Paul van Tilburg 4920ab4abd
Drop Weerplaza precipitation maps (closes: #8) 2022-02-17 21:47:01 +01:00
Paul van Tilburg f67f3dfe82
Only update the cache if retrieval yielded maps
* Add `is_*_stale` methods to the `MapRefresh` trait
* Only update the maps of a type if `retrieve_image` yielded something
  or if the maps are stale
* Also only then bump the timestamp for the map type

This means if there is nothing in the cache, it will retry each refresh
to get something because the timestamp is not bumped until there is
something. Once there are maps, it will only update it and bump the
timestamp if there is an image, that or, it has become stale and it
can be set to `None` and we end up in the initial state.
2022-02-17 21:38:41 +01:00
Paul van Tilburg 8d2717b392
Provide not the first map but an instant-relative map
This calculates which offset to use in the maps series with respect to
the instant of caching. It assumes the first map is current for the
instant it was retrieved.

* Rename `*_first` to `*_at` methods
* For convenience, change the types of `*_MAP_COUNT` to `u32`
* Introduce `*_MAP_INTERVAL` constants to indicate the number of
  seconds each map in the series applies to
* Return `None` if the provided instant is too far in the future
2022-02-17 21:38:16 +01:00
Paul van Tilburg 7061842bd3
Sort Cargo.toml 2022-02-16 22:20:20 +01:00
Paul van Tilburg 88b24a83ff
Move blocking image load to separate task 2022-02-16 22:02:32 +01:00
Paul van Tilburg f4a12dacdb Add a debug handler for marking a position on a map
The map that is used depends on the selected metric.
2022-02-15 17:06:32 +01:00
Paul van Tilburg 9531114eec Add methods to get the first map of each type
Also introduce constants for the number of maps included in a single
`DynamicImage`.
2022-02-15 17:04:04 +01:00
Paul van Tilburg 3a48f234e9 Introduce the Position struct; add position module
* Use `Position` everywhere instead of latitude/longitude float values
* Implement `Partial`, `Eq` and `Hash` for `Position` so it can
  part of a cache key
* Drop the `cache_key` helper function
* Rename the `address_position` function to `resolve_address`
* Add methods on `Position` for formatting latitude/longitude with
  a given precision (used for URL parameters in providers)
2022-02-15 14:15:59 +01:00
Paul van Tilburg b2f63db6b4 Mention open issues in documentation; tweak docs 2022-02-15 14:15:21 +01:00
Paul van Tilburg c76e2315b5 Split off forecast stuff to a separate module 2022-02-15 13:14:01 +01:00
Paul van Tilburg 0c5367f87f
Document caching; increase Luchtmeetnet caching to 30min 2022-02-14 21:40:07 +01:00
Paul van Tilburg 79981314d3
Make output messages more consistent 2022-02-14 21:13:35 +01:00
Paul van Tilburg 8d19dbb517
Implement caching for provider get requests (closes: #2)
* Also cache address geocoding requests to OSM Nomatim!
* Use the `cached` crate for an easy implementation
* Add the `cache_key` helper function to deal with floats being annoying
* Cache Buienradar get request for 5 minutes (per position/metric)
* Cache Luchtmeetnet get request for 5 minutes (per position/metric)
* Note the `Item` structs need to implement `Clone` now because
  the cache will own them and Rocket will want a copy too
2022-02-14 21:06:31 +01:00
Paul van Tilburg 927cb0ad92
Fix geocoded address having latitude/longitude swapped 2022-02-14 21:04:31 +01:00
Paul van Tilburg c231447ce9
Be pedantic with constant number notation 😉 2022-02-13 21:31:12 +01:00
Paul van Tilburg 859288a329
Increase the refresh intervals; improve documentation 2022-02-13 21:24:26 +01:00
Paul van Tilburg 309c79d83c
Disable retrieving precipitation maps for now (see #8) 2022-02-13 21:23:48 +01:00
Paul van Tilburg 576bcc6640
Fix UV index maps base URL
The typo was introduced in commit d432bb4.
2022-02-13 16:55:21 +01:00
Paul van Tilburg f6b26c9659
Fix expect message 2022-02-13 16:55:03 +01:00
Paul van Tilburg 79dac18655
Parse Buienradar as CSV file using serde
* Add the `csv` crate as a dependency
* Use the `Row` struct as intermediate object
* Turn the `parse_value` function into the `convert_value` function that
  cannot fail
2022-02-13 16:46:02 +01:00
Paul van Tilburg 4232263a45
Hook up the Buienradar provider metric in the forecast
Also, sync up the Luchtmeetnet provider documentation a bit.
2022-02-13 15:39:09 +01:00
Paul van Tilburg 6279d379ab
Add the Buienradar provider 2022-02-13 15:38:17 +01:00
Paul van Tilburg d432bb4cd6
Use URL objects instead of formatted strings
Use `request::Url` for this, so we don't have to depend on the `url`
crate ourselves.

Also, make the URL constants more uniform.
2022-02-13 13:10:12 +01:00
Paul van Tilburg 66abc9c4db
Hook up the Luchtmeetnet provider metrics in the forecast 2022-02-13 12:46:42 +01:00
Paul van Tilburg 59c177d508
Add the Luchtmeetnet provider
Also introduce the providers module.
2022-02-13 12:45:27 +01:00
Paul van Tilburg cbd686bd60
Small documentation improvements 2022-02-13 11:22:22 +01:00
Paul van Tilburg cf77dbb5e7
Refactor maps cache to hold lock as short as possible
This makes the response time way more snappy when the maps thread
is updating its cache.

* Move the `MapsHandle` type to the `maps` module
* SWitch to using the standard library mutex
* Split refresh methods into retrieval methods that don't need the lock
  and check timestamp & update methods that only need it shortly
* Introduce the `MapsRefresh` trait and implement it for `MapsHandle`
* Reorder some methods for clarity
* Small documentation fixes
2022-02-13 11:22:02 +01:00
Paul van Tilburg 9b9b1a5f77
Refactor so that no static is necessary for the maps cache
* Replace the lazy `once_cell` by a maps handle type
* Use Rocket's managed state to manage a handle
* Ensure that the handlers have access to it
* Pass another handle to the maps updater loop
* Try to keep the lock as short as possible

Still, long downloads block the lock. Add a FIXME to refactor this
so the lock is only taken when updating the maps fields.
2022-02-12 21:35:58 +01:00
Paul van Tilburg 72fe9577bd
Implement retrieving and caching maps 2022-02-12 21:08:13 +01:00
Paul van Tilburg d058ab4448
Set up a global maps (cache) object 2022-02-12 17:20:36 +01:00
Paul van Tilburg b5dae45868
Create maps cache and run its task next to Rocket 2022-02-12 17:12:06 +01:00
Paul van Tilburg 6b24c4f6e7
Move blocking geocding forward resolving to a separate thread 2022-02-12 17:12:01 +01:00
Paul van Tilburg ae2d2c1c56 Merge pull request 'Clarify that PAQI is the combination of pollen and air quality index' (#1) from admar/sinoptik:clarify-PAQI into main
Reviewed-on: paul/sinoptik#1
2022-02-12 17:10:45 +01:00
20 changed files with 4099 additions and 1036 deletions

View File

@ -0,0 +1,45 @@
name: "Check, lint and test using Cargo"
on:
- pull_request
- push
- workflow_dispatch
jobs:
check_lint:
name: Check, lint and test
runs-on: debian-latest
steps:
- name: Checkout sources
uses: actions/checkout@v3
- name: Install Rust stable toolchain
uses: https://github.com/actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
components: rustfmt, clippy
- name: Run cargo check
uses: https://github.com/actions-rs/cargo@v1
with:
command: check
- name: Run cargo clippy
uses: https://github.com/actions-rs/cargo@v1
with:
command: clippy
args: -- -D warnings
- name: Run cargo fmt
uses: https://github.com/actions-rs/cargo@v1
with:
command: fmt
args: --all -- --check
- name: Run cargo test
uses: https://github.com/actions-rs/cargo@v1
with:
command: test
args: --all-features

View File

@ -0,0 +1,112 @@
name: "Release"
on:
push:
tags:
- "v*"
jobs:
release:
name: "Release"
runs-on: debian-latest
steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Determine the version of the release
run: |
VERSION=${GITHUB_REF_NAME#v}
echo "Releasing version: $VERSION"
echo "VERSION=$VERSION" >> $GITHUB_ENV
- name: Get the release notes from the changelog
run: |
EOF=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
RELEASE_NOTES=$(sed -n -e "/^## \[$VERSION\]/,/^## \[/{//"'!'"p;}" CHANGELOG.md | sed -e '1d;$d')
echo "Release notes:"
echo
echo "$RELEASE_NOTES"
echo "RELEASE_NOTES<<$EOF" >> "$GITHUB_ENV"
echo "$RELEASE_NOTES" >> "$GITHUB_ENV"
echo "$EOF" >> "$GITHUB_ENV"
- name: Install Go
uses: actions/setup-go@v4
with:
go-version: '>=1.20.1'
- name: Release to Gitea
uses: actions/release-action@main
with:
# This is available by default.
api_key: '${{ secrets.RELEASE_TOKEN }}'
files: FIXME
title: 'Release ${{ env.VERSION }}'
body: '${{ env.RELEASE_NOTES }}'
release-crate:
name: "Release Rust crate"
runs-on: debian-latest
steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Install Rust stable toolchain
uses: https://github.com/actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: Use sparse Cargo index for crates.io
run: echo -e '[registries.crates-io]\nprotocol = "sparse"' >> /root/.cargo/config.toml
- name: Register the Gitea crate registry with Cargo
run: echo -e '[registries.luon]\nindex = "https://git.luon.net/paul/_cargo-index.git"' >> /root/.cargo/config.toml
- name: Run cargo publish
uses: https://github.com/actions-rs/cargo@v1
env:
# This needs to be provided for the repository; no login necessary as a result.
CARGO_REGISTRIES_LUON_TOKEN: '${{ secrets.CARGO_TOKEN }}'
with:
command: publish
args: --registry luon
release-deb:
name: "Release Debian package"
runs-on: debian-latest
steps:
- name: Checkout sources
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Install Rust stable toolchain
uses: https://github.com/actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: Install cargo-deb
uses: https://github.com/brndnmtthws/rust-action-cargo-binstall@v1
with:
packages: cargo-deb
- name: Run cargo-deb
uses: https://github.com/actions-rs/cargo@v1
with:
command: deb
- name: Publish Debian package
env:
DEB_REPO_TOKEN: '${{ secrets.DEB_REPO_TOKEN }}'
run: |
curl --config <(printf "user=%s:%s" paul "${DEB_REPO_TOKEN}") \
--upload-file target/debian/sinoptik*.deb \
https://git.luon.net/api/packages/paul/debian/pool/bookworm/main/upload

1
.gitignore vendored
View File

@ -1 +1,2 @@
/target
Rocket.toml

206
CHANGELOG.md Normal file
View File

@ -0,0 +1,206 @@
# Changelog
All notable changes to Sinoptik will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.2.12] - 2024-05-09
### Security
* Updated dependencies, fixes security advisiories:
* [RUSTSEC-2024-0019](https://rustsec.org/advisories/RUSTSEC-2024-0019)
* [RUSTSEC-2024-0332](https://rustsec.org/advisories/RUSTSEC-2024-0332)
### Changed
* Update dependency on `cached`, `chrono-tz`, `image` and `reqwest`
### Fixed
* Fix tests; reduce required accuracy for geocoded coordinates again and
don't run background map updates during tests
## [0.2.11] - 2024-02-27
### Security
* Updated dependencies, fixes security advisories:
* [RUSTSEC-2024-0003](https://rustsec.org/advisories/RUSTSEC-2024-0003)
* [RUSTSEC-2023-0072](https://rustsec.org/advisories/RUSTSEC-2024-0072)
### Fixed
* Fix clippy issue
* Tweak/fix tests; reduce required accuracy for geocoded coordinates
## [0.2.10] - 2023-11-03
### Security
* Update dependencies
([RUSTSEC-2020-0071](https://rustsec.org/advisories/RUSTSEC-2020-0071.html),
[RUSTSEC-2023-0044](https://rustsec.org/advisories/RUSTSEC-2023-0044.html))
### Changed
* Switch to Rocket 0.5 RC4
* Update dependency on `cached`
### Fixed
* Fix clippy issues
## [0.2.9] - 2023-08-25
### Changed
* Update release Gitea Actions workflow; add seperate job to release Debian
package to the new repository
### Security
* Update dependencies ([RUSTSEC-2023-0044](https://rustsec.org/advisories/RUSTSEC-2023-0044))
## [0.2.8] - 2023-06-05
### Added
* Print the version on lift off (#30)
* Add a `/version` endpoint to the API (#30)
### Changed
* Update dependency on `cached`
### Fixed
* Properly attribute the PAQI metric in its description(s)
### Removed
* No longer provide a map for the PAQI metric; the map used is only for pollen
## [0.2.7] - 2023-05-26
### Fixed
* Switch back to the original Buienradar color scheme/maps key (#27)
* Fix the token used to publish the crate to the Cargo package index
## [0.2.6] - 2023-05-24
### Added
* Add full release Gitea Actions workflow
### Changed
* Simplify Gitea Actions check, lint and test workflow
* Improve no known map colors found error description
### Fixed
* Update coordinates of Eindhoven in tests (Nomatim changed its geocoding)
* Increase sampling area to 31×31 pixels (#26)
* Switch to new Buienradar color scheme/maps key (#27)
## [0.2.5] - 2023-03-24
### Added
* Add Gitea Actions workflow for cargo
### Changed
* Updated dependencies on `cached`, `chrono-tz` and `geocoding`
### Fixed
* Fix float comparison in tests
* Fix clippy issues
### Security
* Update dependencies ([RUSTSEC-2023-0018](https://rustsec.org/advisories/RUSTSEC-2023-0018.html))
## [0.2.4] - 2022-07-05
### Added
* Add proper error handling and show them via the API (#25)
### Changed
* Run map refresher as an ad hoc liftoff fairing in Rocket
* Changed emojis in log output
### Removed
* Removed `AQI_max` and `pollen_max` from the forecast JSON introduced in
version 0.2.0
### Fixed
* Verify sample coordinate bounds (#24)
* Default to current time if `Last-Modified` HTTP header is missing for
retrieved maps
## [0.2.3] - 2022-05-21
### Fixed
* Update the examples in `README.md`
* Fix tests by adding missing type
* Fix map key color code for level 8 used by map sampling
## [0.2.2] - 2022-05-10
### Changed
* Switch to Rocket 0.5 RC2
### Fixed
* Fix timestamps for map samples not being correct (AQI, PAQI, UVI metrics) (#22)
* Valid samples/items will no longer be discarded too early
## [0.2.1] - 2022-05-08
### Added
* Add tests for the merge functionality of the combined provider (PAQI)
### Fixed
* Filter out old item/samples in combined provider (PAQI)
## [0.2.0] - 2022-05-07
### Added
* Add `AQI_max` and `pollen_max` to the forecast JSON (only when the PAQI
metric is selected) (#20)
## [0.1.0] - 2022-03-07
Initial release.
[Unreleased]: https://git.luon.net/paul/sinoptik/compare/v0.2.12...HEAD
[0.2.12]: https://git.luon.net/paul/sinoptik/compare/v0.2.11...v0.2.12
[0.2.11]: https://git.luon.net/paul/sinoptik/compare/v0.2.10...v0.2.11
[0.2.10]: https://git.luon.net/paul/sinoptik/compare/v0.2.9...v0.2.10
[0.2.9]: https://git.luon.net/paul/sinoptik/compare/v0.2.8...v0.2.9
[0.2.8]: https://git.luon.net/paul/sinoptik/compare/v0.2.7...v0.2.8
[0.2.7]: https://git.luon.net/paul/sinoptik/compare/v0.2.6...v0.2.7
[0.2.6]: https://git.luon.net/paul/sinoptik/compare/v0.2.5...v0.2.6
[0.2.5]: https://git.luon.net/paul/sinoptik/compare/v0.2.4...v0.2.5
[0.2.4]: https://git.luon.net/paul/sinoptik/compare/v0.2.3...v0.2.4
[0.2.3]: https://git.luon.net/paul/sinoptik/compare/v0.2.2...v0.2.3
[0.2.2]: https://git.luon.net/paul/sinoptik/compare/v0.2.1...v0.2.2
[0.2.1]: https://git.luon.net/paul/sinoptik/compare/v0.2.0...v0.2.1
[0.2.0]: https://git.luon.net/paul/sinoptik/compare/v0.1.0...v0.2.0
[0.1.0]: https://git.luon.net/paul/sinoptik/commits/tag/v0.1.0

2255
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -1,9 +1,67 @@
[package]
name = "sinoptik"
version = "0.1.0"
version = "0.2.12"
authors = [
"Admar Schoonen <admar@luon.net",
"Paul van Tilburg <paul@luon.net>"
]
edition = "2021"
description = "Web service that provides an API for today's weather forecast"
readme = "README.md"
repository = "https://git.luon.net/paul/sinoptik"
license = "MIT"
[dependencies]
cached = { version = "0.51.3", features = ["async"] }
chrono = "0.4.19"
geocoding = "0.3.1"
rocket = { version = "0.5.0-rc.1", features = ["json"] }
chrono-tz = "0.9.0"
csv = "1.1.6"
geocoding = "0.4.0"
image = { version = "0.25.1", default-features = false, features = ["png"]}
reqwest = { version = "0.12.4", features = ["json"] }
rocket = { version = "0.5.0-rc.3", features = ["json"] }
thiserror = "1.0.31"
[build-dependencies]
vergen = { version = "8.2.1", default-features = false, features = ["build", "git", "gitcl"] }
[dev-dependencies]
assert_float_eq = "1.1.3"
assert_matches = "1.5.0"
[package.metadata.deb]
maintainer = "Paul van Tilburg <paul@luon.net>"
copyright = "2022, Paul van Tilburg"
depends = "$auto, systemd"
extended-description = """\
Sinoptik is a (REST) API service that provides an API for today's weather
forecast. It can provide you with a specific set or all available metrics that
it supports.
Currently supported metrics are:
* Air quality index (per hour, from Luchtmeetnet)
* NO concentration (per hour, from Luchtmeetnet)
* O concentration (per hour, from Luchtmeetnet)
* Particulate matter (PM10) concentration (per hour, from Luchtmeetnet)
* Pollen (per hour, from Buienradar)
* Pollen/air quality index (per hour, combined from Buienradar and
Luchtmeetnet)
* Precipitation (per 5 minutes, from Buienradar)
* UV index (per day, from Buienradar)
Because of the currently supported data providers, only data for The
Netherlands can be queried.
"""
section = "net"
priority = "optional"
assets = [
["README.md", "usr/share/doc/sinoptik/", "664"],
["Rocket.toml.example", "/etc/sinoptik.toml", "644"],
["target/release/sinoptik", "usr/sbin/sinoptik", "755"]
]
conf-files = [
"/etc/sinoptik.toml"
]
maintainer-scripts = "debian/"
systemd-units = { unit-name = "sinoptik" }

19
LICENSE Normal file
View File

@ -0,0 +1,19 @@
The MIT License (MIT)
Copyright (c) 2018 Paul van Tilburg
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

248
README.md Normal file
View File

@ -0,0 +1,248 @@
# Sinoptik
Sinoptik is a (REST) API service that provides an API for today's weather
forecast. It can provide you with a specific set or all available metrics
that it supports.
Currently supported metrics are:
* Air quality index (per hour, from [Luchtmeetnet])
* NO₂ concentration (per hour, from [Luchtmeetnet])
* O₃ concentration (per hour, from [Luchtmeetnet])
* Particulate matter (PM10) concentration (per hour, from [Luchtmeetnet])
* Pollen (per hour, from [Buienradar])
* Pollen/air quality index (per hour, combined from [Buienradar] and
[Luchtmeetnet])
* Precipitation (per 5 minutes, from [Buienradar])
* UV index (per day, from [Buienradar])
[Buienradar]: https://buienradar.nl
[Luchtmeetnet]: https://luchtmeetnet.nl
Because of the currently supported data providers, only data for
The Netherlands can be queried.
## Building & running
Using Cargo, it is easy to build and run Sinoptik, just run:
```shell
$ cargo run --release
...
Compiling sinoptik v0.1.0 (/path/to/sinoptik)
Finished release [optimized] target(s) in 9m 26s
Running `/path/to/sinoptik/target/release/sinoptik`
```
(Note that Rocket listens on `127.0.0.1:3000` by default for debug builds, i.e.
builds when you don't add `--release`.)
You can provide Rocket with configuration to use a different address and/or port.
Just create a `Rocket.toml` file that contains (or copy `Rocket.toml.example`):
```toml
[default]
address = "0.0.0.0"
port = 2356
```
This will work independent of the type of build. For more about Rocket's
configuration, see: <https://rocket.rs/v0.5-rc/guide/configuration/>.
## Forecast API endpoint
The `/forecast` API endpoint provides forecasts per requested metric a list of
forecast item which are each comprised of a value and its (UNIX) timestamp. It
does so for a requested location.
### Locations
To select a location, you can either provide an address, or a geocoded position
by providing a latitude and longitude.
For example, to get forecasts for all metrics for the Stationsplein in Utrecht,
use:
```http
GET /forecast?address=Stationsplein,Utrecht&metrics[]=all
```
or directly by using its geocoded position:
```http
GET /forecast?lat=52.0902&lon=5.1114&metrics[]=all
```
### Metrics
When querying, the metrics need to be selected. It can be one of: `AQI`, `NO2`,
`O3`, `PAQI`, `PM10`, `pollen`, `precipitation` or `UVI`. If you use metric
`all`, or `all` is part of the selected metrics, all metrics will be retrieved.
Note that the parameter "array" notation as well as the repeated parameter
notation are supported. For example:
```http
GET /forecast?address=Stationsplein,Utrecht&metrics[]=AQI&metrics[]=pollen
GET /forecast?address=Stationsplein,Utrecht&metrics=AQI&metrics=pollen
GET /forecast?address=Stationsplein,Utrecht&metrics=all
```
### Forecast responses
The response of the API is a JSON object that contains three fixed fields:
* `lat`: the latitude of the geocoded position the forecast is for (number)
* `lon`: the longitude of the geocoded position the forecast is for (number)
* `time`: the (UNIX) timestamp of the forecast, basically "now" (number)
Then, it contains a field per requested metric with a list of forecast items
with two fixed fields as value:
* `time`: the (UNIX) timestamp for that forecasted value (number)
* `value`: the forecasted value for the metric (number)
An example when requesting just UVI (because it's short) for some random
position:
```json
{
"lat": 52.0905169,
"lon": 5.1109709,
"time": 1652188682,
"UVI": [
{
"time": 1652140800,
"value": 4
},
{
"time": 1652227200,
"value": 4
},
{
"time": 1652313600,
"value": 4
},
{
"time": 1652400000,
"value": 4
},
{
"time": 1652486400,
"value": 5
}
]
}
```
#### Combined metric PAQI
The PAQI (pollen/air quality index) metric is a special combined metric.
If selected, it merges items from the AQI and pollen metric into `PAQI` by
selecting the maximum value for each hour:
```json
{
"lat": 52.0905169,
"lon": 5.1109709,
"time": 1652189065,
"PAQI": [
{
"time": 1652187600,
"value": 6.09
},
{
"time": 1652191200,
"value": 6.09
},
...
]
}
```
#### Errors
If geocoding of an address is requested but fails, a not found error is
returned (HTTP 404). with the following body (this will change in the future):
```json
{
"error": {
"code": 404,
"reason": "Not Found",
"description": "The requested resource could not be found."
}
}
```
If for any specific metric an error occurs, the list with forecast items will
be absent. However, the `errors` field will contain the error message for each
failed metric. For example, say Buienradar is down and precipitation forecast
items can not be retrieved:
```json
{
"lat": 52.0905169,
"lon": 5.1109709,
"time": 1654524574,
...
"errors": {
"precipitation": "HTTP request error: error sending request for url (https://gpsgadget.buienradar.nl/data/raintext?lat=52.09&lon=5.11): error trying to connect: tcp connect error: Connection refused (os error 111)"
}
}
```
## Map API endpoint
The `/map` API endpoint basically only exists for debugging purposes. Given an
address or geocoded position, it shows the current map for the provided metric
and draws a crosshair on the position.
Currently, only the `PAQI`, `pollen` and `UVI` metrics are backed by a map.
For example, to get the current pollen map with a crosshair on Stationsplein in
Utrecht, use:
```http
GET /map?address=Stationsplein,Utrecht&metric=pollen
```
or directly by using its geocoded position:
```http
GET /map?lat=52.0902&lon=5.1114&metric=pollen
```
### Map responses
The response is a PNG image with a crosshair drawn on the map. If geocoding of
an address fails or if the position is out of bounds of the map, nothing is
returned (HTTP 404). If the maps cannot/have not been downloaded or cached yet,
a service unavailable error is returned (HTTP 503).
## Version API endpoint
The `/version` API endpoint provides information of the current version and
build of the service. This can be used to check if it needs to be updated.
Again, there is no path and no query parameters, just:
```http
GET /version
```
### Version responses
The response uses the JSON format and typically looks like this:
```json
{
"version": "0.2.7",
"timestamp": "2023-05-29T13:34:34.701323159Z",
"git_sha": "bb5962d",
"git_timestamp": "2023-05-29T15:32:17.000000000+02:00"
}
```
(Build and git information in example output may be out of date.)
## License
Sinoptik is licensed under the MIT license (see the `LICENSE` file or
<http://opensource.org/licenses/MIT>).

3
Rocket.toml.example Normal file
View File

@ -0,0 +1,3 @@
[default]
address = "0.0.0.0"
port = 2356

9
build.rs Normal file
View File

@ -0,0 +1,9 @@
use std::error::Error;
use vergen::EmitBuilder;
fn main() -> Result<(), Box<dyn Error>> {
// Generate the `cargo:` instructions to fill the appropriate environment variables.
EmitBuilder::builder().all_build().all_git().emit()?;
Ok(())
}

45
debian/sinoptik.service vendored Normal file
View File

@ -0,0 +1,45 @@
[Unit]
Description=Sinoptik API web server
After=network.target
[Service]
Type=simple
AmbientCapabilities=
CapabilityBoundingSet=
DynamicUser=yes
LockPersonality=yes
MemoryDenyWriteExecute=yes
NoNewPrivileges=yes
ProtectClock=yes
ProtectControlGroups=yes
ProtectHome=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectSystem=strict
PrivateDevices=yes
PrivateMounts=yes
PrivateTmp=yes
PrivateUsers=yes
RemoveIPC=yes
RestrictAddressFamilies=AF_INET AF_INET6
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM
UMask=0077
ExecStart=/usr/sbin/sinoptik
Restart=on-failure
RestartSec=10
StartLimitInterval=1m
StartLimitBurst=5
Environment="ROCKET_CONFIG=/etc/sinoptik.toml"
[Install]
WantedBy=multi-user.target

218
src/forecast.rs Normal file
View File

@ -0,0 +1,218 @@
//! Forecast retrieval and construction.
//!
//! This module is used to construct a [`Forecast`] for the given position by retrieving data for
//! the requested metrics from their providers.
use std::collections::BTreeMap;
use std::fmt;
use rocket::serde::Serialize;
use crate::maps::MapsHandle;
use crate::position::Position;
use crate::providers::buienradar::{Item as BuienradarItem, Sample as BuienradarSample};
use crate::providers::combined::Item as CombinedItem;
use crate::providers::luchtmeetnet::Item as LuchtmeetnetItem;
use crate::{providers, Error};
/// The current forecast for a specific location.
///
/// Only the metrics asked for are included as well as the position and current time.
#[derive(Debug, Default, Serialize)]
#[serde(crate = "rocket::serde")]
pub(crate) struct Forecast {
/// The latitude of the position.
lat: f64,
/// The longitude of the position.
lon: f64,
/// The current time (in seconds since the UNIX epoch).
time: i64,
/// The air quality index (when asked for).
#[serde(rename = "AQI", skip_serializing_if = "Option::is_none")]
aqi: Option<Vec<LuchtmeetnetItem>>,
/// The NO₂ concentration (when asked for).
#[serde(rename = "NO2", skip_serializing_if = "Option::is_none")]
no2: Option<Vec<LuchtmeetnetItem>>,
/// The O₃ concentration (when asked for).
#[serde(rename = "O3", skip_serializing_if = "Option::is_none")]
o3: Option<Vec<LuchtmeetnetItem>>,
/// The combination of pollen + air quality index (when asked for).
#[serde(rename = "PAQI", skip_serializing_if = "Option::is_none")]
paqi: Option<Vec<CombinedItem>>,
/// The particulate matter in the air (when asked for).
#[serde(rename = "PM10", skip_serializing_if = "Option::is_none")]
pm10: Option<Vec<LuchtmeetnetItem>>,
/// The pollen in the air (when asked for).
#[serde(skip_serializing_if = "Option::is_none")]
pollen: Option<Vec<BuienradarSample>>,
/// The precipitation (when asked for).
#[serde(skip_serializing_if = "Option::is_none")]
precipitation: Option<Vec<BuienradarItem>>,
/// The UV index (when asked for).
#[serde(rename = "UVI", skip_serializing_if = "Option::is_none")]
uvi: Option<Vec<BuienradarSample>>,
/// Any errors that occurred.
#[serde(skip_serializing_if = "BTreeMap::is_empty")]
errors: BTreeMap<Metric, String>,
}
impl Forecast {
fn new(position: Position) -> Self {
Self {
lat: position.lat,
lon: position.lon,
time: chrono::Utc::now().timestamp(),
..Default::default()
}
}
fn log_error(&mut self, metric: Metric, error: Error) {
eprintln!("💥 Encountered error during forecast: {}", error);
self.errors.insert(metric, error.to_string());
}
}
/// The supported forecast metrics.
///
/// This is used for selecting which metrics should be calculated & returned.
#[allow(clippy::upper_case_acronyms)]
#[derive(
Copy, Clone, Debug, Eq, Hash, Ord, PartialOrd, PartialEq, Serialize, rocket::FromFormField,
)]
#[serde(crate = "rocket::serde")]
pub(crate) enum Metric {
/// All metrics.
#[field(value = "all")]
All,
/// The air quality index.
AQI,
/// The NO₂ concentration.
NO2,
/// The O₃ concentration.
O3,
/// The combination of pollen + air quality index.
PAQI,
/// The particulate matter in the air.
PM10,
/// The pollen in the air.
#[serde(rename(serialize = "pollen"))]
Pollen,
#[serde(rename(serialize = "precipitation"))]
/// The precipitation.
Precipitation,
/// The UV index.
UVI,
}
impl Metric {
/// Returns all supported metrics.
fn all() -> Vec<Metric> {
use Metric::*;
Vec::from([AQI, NO2, O3, PAQI, PM10, Pollen, Precipitation, UVI])
}
}
impl fmt::Display for Metric {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Metric::All => write!(f, "All"),
Metric::AQI => write!(f, "AQI"),
Metric::NO2 => write!(f, "NO2"),
Metric::O3 => write!(f, "O3"),
Metric::PAQI => write!(f, "PAQI"),
Metric::PM10 => write!(f, "PM10"),
Metric::Pollen => write!(f, "pollen"),
Metric::Precipitation => write!(f, "precipitation"),
Metric::UVI => write!(f, "UVI"),
}
}
}
/// Calculates and returns the forecast.
///
/// The provided list `metrics` determines what will be included in the forecast.
pub(crate) async fn forecast(
position: Position,
metrics: Vec<Metric>,
maps_handle: &MapsHandle,
) -> Forecast {
let mut forecast = Forecast::new(position);
// Expand the `All` metric if present, deduplicate otherwise.
let mut metrics = metrics;
if metrics.contains(&Metric::All) {
metrics = Metric::all();
} else {
metrics.dedup()
}
for metric in metrics {
match metric {
// This should have been expanded to all the metrics matched below.
Metric::All => unreachable!("The all metric should have been expanded"),
Metric::AQI => {
forecast.aqi = providers::luchtmeetnet::get(position, metric)
.await
.map_err(|err| forecast.log_error(metric, err))
.ok()
}
Metric::NO2 => {
forecast.no2 = providers::luchtmeetnet::get(position, metric)
.await
.map_err(|err| forecast.log_error(metric, err))
.ok()
}
Metric::O3 => {
forecast.o3 = providers::luchtmeetnet::get(position, metric)
.await
.map_err(|err| forecast.log_error(metric, err))
.ok()
}
Metric::PAQI => {
forecast.paqi = providers::combined::get(position, metric, maps_handle)
.await
.map_err(|err| forecast.log_error(metric, err))
.ok()
}
Metric::PM10 => {
forecast.pm10 = providers::luchtmeetnet::get(position, metric)
.await
.map_err(|err| forecast.log_error(metric, err))
.ok()
}
Metric::Pollen => {
forecast.pollen = providers::buienradar::get_samples(position, metric, maps_handle)
.await
.map_err(|err| forecast.log_error(metric, err))
.ok()
}
Metric::Precipitation => {
forecast.precipitation = providers::buienradar::get_items(position, metric)
.await
.map_err(|err| forecast.log_error(metric, err))
.ok()
}
Metric::UVI => {
forecast.uvi = providers::buienradar::get_samples(position, metric, maps_handle)
.await
.map_err(|err| forecast.log_error(metric, err))
.ok()
}
}
}
forecast
}

410
src/lib.rs Normal file
View File

@ -0,0 +1,410 @@
#![doc = include_str!("../README.md")]
#![warn(
clippy::all,
missing_copy_implementations,
missing_debug_implementations,
rust_2018_idioms,
rustdoc::broken_intra_doc_links,
trivial_casts,
trivial_numeric_casts,
renamed_and_removed_lints,
unsafe_code,
unstable_features,
unused_import_braces,
unused_qualifications
)]
#![deny(missing_docs)]
use std::sync::{Arc, Mutex};
use rocket::fairing::AdHoc;
use rocket::http::Status;
use rocket::response::Responder;
use rocket::serde::json::Json;
use rocket::serde::Serialize;
use rocket::{get, routes, Build, Request, Rocket, State};
use self::forecast::{forecast, Forecast, Metric};
use self::maps::{mark_map, Error as MapsError, Maps, MapsHandle};
use self::position::{resolve_address, Position};
pub(crate) mod forecast;
pub(crate) mod maps;
pub(crate) mod position;
pub(crate) mod providers;
/// The possible provider errors that can occur.
#[derive(Debug, thiserror::Error)]
pub(crate) enum Error {
/// A CSV parse error occurred.
#[error("CSV parse error: {0}")]
CsvParse(#[from] csv::Error),
/// A geocoding error occurred.
#[error("Geocoding error: {0}")]
Geocoding(#[from] geocoding::GeocodingError),
/// An HTTP request error occurred.
#[error("HTTP request error: {0}")]
HttpRequest(#[from] reqwest::Error),
/// Failed to join a task.
#[error("Failed to join a task: {0}")]
Join(#[from] rocket::tokio::task::JoinError),
/// Failed to merge AQI & pollen items.
#[error("Failed to merge AQI & pollen items: {0}")]
Merge(#[from] providers::combined::MergeError),
/// Failed to retrieve or sample the maps.
#[error("Failed to retrieve or sample the maps: {0}")]
Maps(#[from] maps::Error),
/// No geocoded position could be found.
#[error("No geocoded position could be found")]
NoPositionFound,
/// Encountered an unsupported metric.
#[error("Encountered an unsupported metric: {0}")]
UnsupportedMetric(Metric),
}
impl<'r, 'o: 'r> Responder<'r, 'o> for Error {
fn respond_to(self, _request: &'r Request<'_>) -> rocket::response::Result<'o> {
eprintln!("💥 Encountered error during request: {}", self);
let status = match self {
Error::NoPositionFound => Status::NotFound,
Error::Maps(MapsError::NoMapsYet) => Status::ServiceUnavailable,
Error::Maps(MapsError::OutOfBoundCoords(_, _)) => Status::NotFound,
Error::Maps(MapsError::OutOfBoundOffset(_)) => Status::NotFound,
_ => Status::InternalServerError,
};
Err(status)
}
}
#[derive(Responder)]
#[response(content_type = "image/png")]
struct PngImageData(Vec<u8>);
/// Result type that defaults to [`Error`] as the default error type.
pub(crate) type Result<T, E = Error> = std::result::Result<T, E>;
/// The version information as JSON response.
#[derive(Debug, Serialize)]
#[serde(crate = "rocket::serde")]
struct VersionInfo {
/// The version of the build.
version: String,
/// The timestamp of the build.
timestamp: String,
/// The (most recent) git SHA used for the build.
git_sha: String,
/// The timestamp of the last git commit used for the build.
git_timestamp: String,
}
impl VersionInfo {
/// Retrieves the version information from the environment variables.
fn new() -> Self {
Self {
version: String::from(env!("CARGO_PKG_VERSION")),
timestamp: String::from(env!("VERGEN_BUILD_TIMESTAMP")),
git_sha: String::from(&env!("VERGEN_GIT_SHA")[0..7]),
git_timestamp: String::from(env!("VERGEN_GIT_COMMIT_TIMESTAMP")),
}
}
}
/// Handler for retrieving the forecast for an address.
#[get("/forecast?<address>&<metrics>")]
async fn forecast_address(
address: String,
metrics: Vec<Metric>,
maps_handle: &State<MapsHandle>,
) -> Result<Json<Forecast>> {
let position = resolve_address(address).await?;
let forecast = forecast(position, metrics, maps_handle).await;
Ok(Json(forecast))
}
/// Handler for retrieving the forecast for a geocoded position.
#[get("/forecast?<lat>&<lon>&<metrics>", rank = 2)]
async fn forecast_geo(
lat: f64,
lon: f64,
metrics: Vec<Metric>,
maps_handle: &State<MapsHandle>,
) -> Json<Forecast> {
let position = Position::new(lat, lon);
let forecast = forecast(position, metrics, maps_handle).await;
Json(forecast)
}
/// Handler for showing the current map with the geocoded position of an address for a specific
/// metric.
///
/// Note: This handler is mosly used for debugging purposes!
#[get("/map?<address>&<metric>")]
async fn map_address(
address: String,
metric: Metric,
maps_handle: &State<MapsHandle>,
) -> Result<PngImageData> {
let position = resolve_address(address).await?;
let image_data = mark_map(position, metric, maps_handle).await;
image_data.map(PngImageData)
}
/// Handler for showing the current map with the geocoded position for a specific metric.
///
/// Note: This handler is mosly used for debugging purposes!
#[get("/map?<lat>&<lon>&<metric>", rank = 2)]
async fn map_geo(
lat: f64,
lon: f64,
metric: Metric,
maps_handle: &State<MapsHandle>,
) -> Result<PngImageData> {
let position = Position::new(lat, lon);
let image_data = mark_map(position, metric, maps_handle).await;
image_data.map(PngImageData)
}
/// Returns the version information.
#[get("/version", format = "application/json")]
async fn version() -> Result<Json<VersionInfo>> {
Ok(Json(VersionInfo::new()))
}
/// Sets up Rocket without fairings.
fn rocket_core(maps_handle: MapsHandle) -> Rocket<Build> {
rocket::build()
.mount(
"/",
routes![
forecast_address,
forecast_geo,
map_address,
map_geo,
version
],
)
.manage(maps_handle)
}
/// Sets up Rocket.
fn rocket(maps_handle: MapsHandle) -> Rocket<Build> {
let rocket = rocket_core(Arc::clone(&maps_handle));
let maps_refresher = maps::run(maps_handle);
rocket
.attach(AdHoc::on_liftoff("Maps refresher", |_| {
Box::pin(async move {
// We don't care about the join handle nor error results?
let _refresher = rocket::tokio::spawn(maps_refresher);
})
}))
.attach(AdHoc::on_liftoff("Version", |_| {
Box::pin(async move {
let name = env!("CARGO_PKG_NAME");
let version = env!("CARGO_PKG_VERSION");
let git_sha = &env!("VERGEN_GIT_SHA")[0..7];
println!("🌁 Started {name} v{version} (git @{git_sha})");
})
}))
}
/// Sets up Rocket and the maps cache refresher task.
pub fn setup() -> Rocket<Build> {
let maps = Maps::new();
let maps_handle = Arc::new(Mutex::new(maps));
rocket(maps_handle)
}
#[cfg(test)]
mod tests {
use assert_float_eq::*;
use assert_matches::assert_matches;
use image::{DynamicImage, Rgba, RgbaImage};
use rocket::http::{ContentType, Status};
use rocket::local::blocking::Client;
use rocket::serde::json::Value as JsonValue;
use super::maps::RetrievedMaps;
use super::*;
fn maps_stub(map_count: u32) -> RetrievedMaps {
let map_color = Rgba::from([73, 218, 33, 255]); // First color from map key.
let image =
DynamicImage::ImageRgba8(RgbaImage::from_pixel(820 * map_count, 988, map_color));
RetrievedMaps::new(image)
}
fn maps_handle_stub() -> MapsHandle {
let mut maps = Maps::new();
maps.pollen = Some(maps_stub(24));
maps.uvi = Some(maps_stub(5));
Arc::new(Mutex::new(maps))
}
#[test]
fn forecast_address() {
let maps_handle = maps_handle_stub();
let client = Client::tracked(rocket(maps_handle)).expect("Not a valid Rocket instance");
// Get an empty forecast for the provided address.
let response = client.get("/forecast?address=eindhoven").dispatch();
assert_eq!(response.status(), Status::Ok);
let json = response.into_json::<JsonValue>().expect("Not valid JSON");
assert_float_absolute_eq!(json["lat"].as_f64().unwrap(), 51.448557, 1e-1);
assert_float_absolute_eq!(json["lon"].as_f64().unwrap(), 5.450123, 1e-1);
assert_matches!(json["time"], JsonValue::Number(_));
assert_matches!(json.get("AQI"), None);
assert_matches!(json.get("NO2"), None);
assert_matches!(json.get("O3"), None);
assert_matches!(json.get("PAQI"), None);
assert_matches!(json.get("PM10"), None);
assert_matches!(json.get("pollen"), None);
assert_matches!(json.get("precipitation"), None);
assert_matches!(json.get("UVI"), None);
// Get a forecast with all metrics for the provided address.
let response = client
.get("/forecast?address=eindhoven&metrics=all")
.dispatch();
assert_eq!(response.status(), Status::Ok);
let json = response.into_json::<JsonValue>().expect("Not valid JSON");
assert_float_absolute_eq!(json["lat"].as_f64().unwrap(), 51.448557, 1e-1);
assert_float_absolute_eq!(json["lon"].as_f64().unwrap(), 5.450123, 1e-1);
assert_matches!(json["time"], JsonValue::Number(_));
assert_matches!(json.get("AQI"), Some(JsonValue::Array(_)));
assert_matches!(json.get("NO2"), Some(JsonValue::Array(_)));
assert_matches!(json.get("O3"), Some(JsonValue::Array(_)));
assert_matches!(json.get("PAQI"), Some(JsonValue::Array(_)));
assert_matches!(json.get("PM10"), Some(JsonValue::Array(_)));
assert_matches!(json.get("pollen"), Some(JsonValue::Array(_)));
assert_matches!(json.get("precipitation"), Some(JsonValue::Array(_)));
assert_matches!(json.get("UVI"), Some(JsonValue::Array(_)));
}
#[test]
fn forecast_geo() {
let maps_handle = maps_handle_stub();
let client = Client::tracked(rocket(maps_handle)).expect("valid Rocket instance");
// Get an empty forecast for the geocoded location.
let response = client.get("/forecast?lat=51.4&lon=5.5").dispatch();
assert_eq!(response.status(), Status::Ok);
let json = response.into_json::<JsonValue>().expect("Not valid JSON");
assert_f64_near!(json["lat"].as_f64().unwrap(), 51.4);
assert_f64_near!(json["lon"].as_f64().unwrap(), 5.5);
assert_matches!(json["time"], JsonValue::Number(_));
assert_matches!(json.get("AQI"), None);
assert_matches!(json.get("NO2"), None);
assert_matches!(json.get("O3"), None);
assert_matches!(json.get("PAQI"), None);
assert_matches!(json.get("PM10"), None);
assert_matches!(json.get("pollen"), None);
assert_matches!(json.get("precipitation"), None);
assert_matches!(json.get("UVI"), None);
// Get a forecast with all metrics for the geocoded location.
let response = client
.get("/forecast?lat=51.4&lon=5.5&metrics=all")
.dispatch();
assert_eq!(response.status(), Status::Ok);
let json = response.into_json::<JsonValue>().expect("Not valid JSON");
assert_f64_near!(json["lat"].as_f64().unwrap(), 51.4);
assert_f64_near!(json["lon"].as_f64().unwrap(), 5.5);
assert_matches!(json["time"], JsonValue::Number(_));
assert_matches!(json.get("AQI"), Some(JsonValue::Array(_)));
assert_matches!(json.get("NO2"), Some(JsonValue::Array(_)));
assert_matches!(json.get("O3"), Some(JsonValue::Array(_)));
assert_matches!(json.get("PAQI"), Some(JsonValue::Array(_)));
assert_matches!(json.get("PM10"), Some(JsonValue::Array(_)));
assert_matches!(json.get("pollen"), Some(JsonValue::Array(_)));
assert_matches!(json.get("precipitation"), Some(JsonValue::Array(_)));
assert_matches!(json.get("UVI"), Some(JsonValue::Array(_)));
}
#[test]
fn map_address() {
let maps_handle = Arc::new(Mutex::new(Maps::new()));
let maps_handle_clone = Arc::clone(&maps_handle);
let client =
Client::tracked(rocket_core(maps_handle)).expect("Not a valid Rocket instance");
// No maps available yet.
let response = client
.get("/map?address=eindhoven&metric=pollen")
.dispatch();
assert_eq!(response.status(), Status::ServiceUnavailable);
// Load some dummy map.
let mut maps = maps_handle_clone
.lock()
.expect("Maps handle mutex was poisoned");
maps.pollen = Some(maps_stub(24));
drop(maps);
// There should be a map now.
let response = client
.get("/map?address=eindhoven&metric=pollen")
.dispatch();
assert_eq!(response.status(), Status::Ok);
assert_eq!(response.content_type(), Some(ContentType::PNG));
// ... but not if it is out of bounds.
let response = client.get("/map?address=berlin&metric=pollen").dispatch();
assert_eq!(response.status(), Status::NotFound);
// No metric selected, don't know which map to show?
let response = client.get("/map?address=eindhoven").dispatch();
assert_eq!(response.status(), Status::UnprocessableEntity);
}
#[test]
fn map_geo() {
let maps_handle = Arc::new(Mutex::new(Maps::new()));
let maps_handle_clone = Arc::clone(&maps_handle);
let client =
Client::tracked(rocket_core(maps_handle)).expect("Not a valid Rocket instance");
// No maps available yet.
let response = client.get("/map?lat=51.4&lon=5.5&metric=pollen").dispatch();
assert_eq!(response.status(), Status::ServiceUnavailable);
// Load some dummy map.
let mut maps = maps_handle_clone
.lock()
.expect("Maps handle mutex was poisoned");
maps.pollen = Some(maps_stub(24));
drop(maps);
// There should be a map now.
let response = client.get("/map?lat=51.4&lon=5.5&metric=pollen").dispatch();
assert_eq!(response.status(), Status::Ok);
assert_eq!(response.content_type(), Some(ContentType::PNG));
// ... but not if it is out of bounds.
let response = client.get("/map?lat=0.0&lon=0.0&metric=pollen").dispatch();
assert_eq!(response.status(), Status::NotFound);
// No metric passed, don't know which map to show?
let response = client.get("/map?lat=51.4&lon=5.5").dispatch();
assert_eq!(response.status(), Status::UnprocessableEntity);
}
}

View File

@ -1,180 +1,22 @@
//! Service that provides today's weather forecast for air quality, rain and UV metrics.
//!
//! This is useful if you want to prepare for going outside and need to know what happens in the
//! near future or later today.
#![doc = include_str!("../README.md")]
#![warn(
clippy::all,
missing_copy_implementations,
missing_debug_implementations,
rust_2018_idioms,
rustdoc::broken_intra_doc_links
rustdoc::broken_intra_doc_links,
trivial_casts,
trivial_numeric_casts,
renamed_and_removed_lints,
unsafe_code,
unstable_features,
unused_import_braces,
unused_qualifications
)]
#![deny(missing_docs)]
use geocoding::{Forward, Openstreetmap, Point};
use rocket::serde::json::Json;
use rocket::serde::Serialize;
use rocket::{get, launch, routes, FromFormField};
/// The current for a specific location.
///
/// Only the metrics asked for are included as well as the position and current time.
///
/// TODO: Fill the metrics with actual data!
#[derive(Debug, Default, PartialEq, Serialize)]
#[serde(crate = "rocket::serde")]
struct Forecast {
/// The latitude of the position.
lat: f64,
/// The longitude of the position.
lon: f64,
/// The current time (in seconds since the UNIX epoch).
time: i64,
/// The air quality index (when asked for).
#[serde(rename = "AQI", skip_serializing_if = "Option::is_none")]
aqi: Option<u8>,
/// The NO₂ concentration (when asked for).
#[serde(rename = "NO2", skip_serializing_if = "Option::is_none")]
no2: Option<u8>,
/// The O₃ concentration (when asked for).
#[serde(rename = "O3", skip_serializing_if = "Option::is_none")]
o3: Option<u8>,
/// The combination of pollen + air quality index (when asked for).
#[serde(rename = "PAQI", skip_serializing_if = "Option::is_none")]
paqi: Option<u8>,
/// The particulate matter in the air (when asked for).
#[serde(rename = "PM10", skip_serializing_if = "Option::is_none")]
pm10: Option<u8>,
/// The pollen in the air (when asked for).
#[serde(skip_serializing_if = "Option::is_none")]
pollen: Option<u8>,
/// The precipitation (when asked for).
#[serde(skip_serializing_if = "Option::is_none")]
precipitation: Option<u8>,
/// The UV index (when asked for).
#[serde(rename = "UVI", skip_serializing_if = "Option::is_none")]
uvi: Option<u8>,
}
impl Forecast {
fn new(lat: f64, lon: f64) -> Self {
let time = chrono::Utc::now().timestamp();
Self {
lat,
lon,
time,
..Default::default()
}
}
}
/// The supported metrics.
///
/// This is used for selecting which metrics should be calculated & returned.
#[allow(clippy::upper_case_acronyms)]
#[derive(Copy, Clone, Debug, Eq, PartialEq, FromFormField)]
enum Metric {
/// All metrics.
#[field(value = "all")]
All,
/// The air quality index.
AQI,
/// The NO₂ concentration.
NO2,
/// The O₃ concentration.
O3,
/// The combination of pollen + air quality index.
PAQI,
/// The particulate matter in the air.
PM10,
/// The pollen in the air.
Pollen,
/// The precipitation.
Precipitation,
/// The UV index.
UVI,
}
impl Metric {
/// Returns all supported metrics.
fn all() -> Vec<Metric> {
use Metric::*;
Vec::from([AQI, NO2, O3, PAQI, PM10, Pollen, Precipitation, UVI])
}
}
/// Calculates and returns the forecast.
///
/// The provided list `metrics` determines what will be included in the forecast.
async fn forecast(lat: f64, lon: f64, metrics: Vec<Metric>) -> Forecast {
let mut forecast = Forecast::new(lat, lon);
// Expand the `All` metric if present, deduplicate otherwise.
let mut metrics = metrics;
if metrics.contains(&Metric::All) {
metrics = Metric::all();
} else {
metrics.dedup()
}
for metric in metrics {
match metric {
// This should have been expanded to all the metrics matched below.
Metric::All => unreachable!("should have been expanded"),
Metric::AQI => forecast.aqi = Some(1),
Metric::NO2 => forecast.no2 = Some(2),
Metric::O3 => forecast.o3 = Some(3),
Metric::PAQI => forecast.paqi = Some(4),
Metric::PM10 => forecast.pm10 = Some(5),
Metric::Pollen => forecast.pollen = Some(6),
Metric::Precipitation => forecast.precipitation = Some(7),
Metric::UVI => forecast.uvi = Some(8),
}
}
forecast
}
/// Retrieves the geocoded position for the given address.
async fn address_position(address: &str) -> Option<(f64, f64)> {
let osm = Openstreetmap::new();
// FIXME: Handle or log the error.
let points: Vec<Point<f64>> = osm.forward(address).ok()?;
points.get(0).map(|point| (point.x(), point.y()))
}
/// Handler for retrieving the forecast for an address.
#[get("/forecast?<address>&<metrics>")]
async fn forecast_address(address: String, metrics: Vec<Metric>) -> Option<Json<Forecast>> {
let (lat, lon) = address_position(&address).await?;
let forecast = forecast(lat, lon, metrics).await;
Some(Json(forecast))
}
/// Handler for retrieving the forecast for a geocoded position.
#[get("/forecast?<lat>&<lon>&<metrics>", rank = 2)]
async fn forecast_geo(lat: f64, lon: f64, metrics: Vec<Metric>) -> Json<Forecast> {
let forecast = forecast(lat, lon, metrics).await;
Json(forecast)
}
/// Launches rocket.
#[launch]
/// Starts the main maps refresh task and sets up and launches Rocket.
#[rocket::launch]
async fn rocket() -> _ {
rocket::build().mount("/", routes![forecast_address, forecast_geo])
sinoptik::setup()
}

613
src/maps.rs Normal file
View File

@ -0,0 +1,613 @@
//! Maps retrieval and caching.
//!
//! This module provides a task that keeps maps up-to-date using a maps-specific refresh interval.
//! It stores all the maps as [`DynamicImage`]s in memory.
use std::collections::HashMap;
use std::f64::consts::PI;
use std::sync::{Arc, Mutex};
use chrono::serde::ts_seconds;
use chrono::{DateTime, Duration, NaiveDateTime, TimeZone, Utc};
use image::{
DynamicImage, GenericImage, GenericImageView, ImageError, ImageFormat, Pixel, Rgb, Rgba,
};
use reqwest::Url;
use rocket::serde::Serialize;
use rocket::tokio;
use rocket::tokio::time::sleep;
use crate::forecast::Metric;
use crate::position::Position;
/// The possible maps errors that can occur.
#[derive(Debug, thiserror::Error)]
pub(crate) enum Error {
/// A timestamp parse error occurred.
#[error("Timestamp parse error: {0}")]
ChronoParse(#[from] chrono::ParseError),
/// A HTTP request error occurred.
#[error("HTTP request error: {0}")]
HttpRequest(#[from] reqwest::Error),
/// Failed to represent HTTP header as a string.
#[error("Failed to represent HTTP header as a string")]
HttpHeaderToStr(#[from] reqwest::header::ToStrError),
/// An image error occurred.
#[error("Image error: {0}")]
Image(#[from] ImageError),
/// Encountered an invalid image file path.
#[error("Invalid image file path: {0}")]
InvalidImagePath(String),
/// Failed to join a task.
#[error("Failed to join a task: {0}")]
Join(#[from] tokio::task::JoinError),
/// Did not find any known (map key) colors in samples.
#[error("Did not find any known colors in samples")]
NoKnownColorsInSamples,
/// No maps found (yet).
#[error("No maps found (yet)")]
NoMapsYet,
/// Got out of bound coordinates for a map.
#[error("Got out of bound coordinates for a map: ({0}, {1})")]
OutOfBoundCoords(u32, u32),
/// Got out of bound offset for a map.
#[error("Got out of bound offset for a map: {0}")]
OutOfBoundOffset(u32),
}
/// Result type that defaults to [`Error`] as the default error type.
pub(crate) type Result<T, E = Error> = std::result::Result<T, E>;
/// A handle to access the in-memory cached maps.
pub(crate) type MapsHandle = Arc<Mutex<Maps>>;
/// A histogram mapping map key colors to occurences/counts.
type MapKeyHistogram = HashMap<Rgb<u8>, u32>;
/// The Buienradar map key used for determining the score of a coordinate by mapping its color.
///
/// Note that the actual score starts from 1, not 0 as per this array.
#[rustfmt::skip]
const MAP_KEY: [[u8; 3]; 10] = [
[0x49, 0xDA, 0x21], // #49DA21
[0x30, 0xD2, 0x00], // #30D200
[0xFF, 0xF8, 0x8B], // #FFF88B
[0xFF, 0xF6, 0x42], // #FFF642
[0xFD, 0xBB, 0x31], // #FDBB31
[0xFD, 0x8E, 0x24], // #FD8E24
[0xFC, 0x10, 0x3E], // #FC103E
[0x97, 0x0A, 0x33], // #970A33
[0xA6, 0x6D, 0xBC], // #A66DBC
[0xB3, 0x30, 0xA1], // #B330A1
];
/// The Buienradar map sample size.
///
/// Determines the number of pixels in width/height that is sampled around the sampling coordinate.
const MAP_SAMPLE_SIZE: [u32; 2] = [31, 31];
/// The interval between map refreshes (in seconds).
const REFRESH_INTERVAL: tokio::time::Duration = tokio::time::Duration::from_secs(60);
/// The base URL for retrieving the pollen maps from Buienradar.
const POLLEN_BASE_URL: &str =
"https://image.buienradar.nl/2.0/image/sprite/WeatherMapPollenRadarHourlyNL\
?width=820&height=988&extension=png&renderBackground=False&renderBranding=False\
&renderText=False&history=0&forecast=24&skip=0";
/// The interval for retrieving pollen maps.
///
/// The endpoint provides a map for every hour, 24 in total.
const POLLEN_INTERVAL: i64 = 3_600;
/// The number of pollen maps retained.
const POLLEN_MAP_COUNT: u32 = 24;
/// The number of seconds each pollen map is for.
const POLLEN_MAP_INTERVAL: i64 = 3_600;
/// The position reference points for the pollen map.
///
/// Maps the gecoded positions of two reference points as follows:
/// * Latitude and longitude of Vlissingen to its y- and x-position
/// * Latitude of Lauwersoog to its y-position and longitude of Enschede to its x-position
const POLLEN_MAP_REF_POINTS: [(Position, (u32, u32)); 2] = [
(Position::new(51.44, 3.57), (745, 84)), // Vlissingen
(Position::new(53.40, 6.90), (111, 694)), // Lauwersoog (lat/y) and Enschede (lon/x)
];
/// The base URL for retrieving the UV index maps from Buienradar.
const UVI_BASE_URL: &str = "https://image.buienradar.nl/2.0/image/sprite/WeatherMapUVIndexNL\
?width=820&height=988&extension=png&&renderBackground=False&renderBranding=False\
&renderText=False&history=0&forecast=5&skip=0";
/// The interval for retrieving UV index maps.
///
/// The endpoint provides a map for every day, 5 in total.
const UVI_INTERVAL: i64 = 24 * 3_600;
/// The number of UV index maps retained.
const UVI_MAP_COUNT: u32 = 5;
/// The number of seconds each UV index map is for.
const UVI_MAP_INTERVAL: i64 = 24 * 3_600;
/// The position reference points for the UV index map.
const UVI_MAP_REF_POINTS: [(Position, (u32, u32)); 2] = POLLEN_MAP_REF_POINTS;
/// The `MapsRefresh` trait is used to reduce the time a lock needs to be held when updating maps.
///
/// When refreshing maps, the lock only needs to be held when checking whether a refresh is
/// necessary and when the new maps have been retrieved and can be updated.
trait MapsRefresh {
/// Determines whether the pollen maps need to be refreshed.
fn needs_pollen_refresh(&self) -> bool;
/// Determines whether the UV index maps need to be refreshed.
fn needs_uvi_refresh(&self) -> bool;
/// Determines whether the pollen maps are stale.
fn is_pollen_stale(&self) -> bool;
/// Determines whether the UV index maps are stale.
fn is_uvi_stale(&self) -> bool;
/// Updates the pollen maps.
fn set_pollen(&self, result: Result<RetrievedMaps>);
/// Updates the UV index maps.
fn set_uvi(&self, result: Result<RetrievedMaps>);
}
/// Container type for all in-memory cached maps.
#[derive(Debug, Default)]
pub(crate) struct Maps {
/// The pollen maps (from Buienradar).
pub(crate) pollen: Option<RetrievedMaps>,
/// The UV index maps (from Buienradar).
pub(crate) uvi: Option<RetrievedMaps>,
}
impl Maps {
/// Creates a new maps cache.
///
/// It contains an [`DynamicImage`] per maps type, if downloaded, and the timestamp of the last
/// update.
pub(crate) fn new() -> Self {
Self {
pollen: None,
uvi: None,
}
}
/// Returns a current pollen map that marks the provided position.
pub(crate) fn pollen_mark(&self, position: Position) -> Result<DynamicImage> {
let maps = self.pollen.as_ref().ok_or(Error::NoMapsYet)?;
let image = &maps.image;
let stamp = maps.timestamp_base;
let marked_image = map_at(
image,
stamp,
POLLEN_MAP_INTERVAL,
POLLEN_MAP_COUNT,
Utc::now(),
)?;
let coords = project(&marked_image, POLLEN_MAP_REF_POINTS, position)?;
Ok(mark(marked_image, coords))
}
/// Samples the pollen maps for the given position.
pub(crate) fn pollen_samples(&self, position: Position) -> Result<Vec<Sample>> {
let maps = self.pollen.as_ref().ok_or(Error::NoMapsYet)?;
let image = &maps.image;
let map = image.view(0, 0, image.width() / UVI_MAP_COUNT, image.height());
let coords = project(&*map, POLLEN_MAP_REF_POINTS, position)?;
let stamp = maps.timestamp_base;
sample(image, stamp, POLLEN_MAP_INTERVAL, POLLEN_MAP_COUNT, coords)
}
/// Returns a current UV index map that marks the provided position.
pub(crate) fn uvi_mark(&self, position: Position) -> Result<DynamicImage> {
let maps = self.uvi.as_ref().ok_or(Error::NoMapsYet)?;
let image = &maps.image;
let stamp = maps.timestamp_base;
let marked_image = map_at(image, stamp, UVI_MAP_INTERVAL, UVI_MAP_COUNT, Utc::now())?;
let coords = project(&marked_image, POLLEN_MAP_REF_POINTS, position)?;
Ok(mark(marked_image, coords))
}
/// Samples the UV index maps for the given position.
pub(crate) fn uvi_samples(&self, position: Position) -> Result<Vec<Sample>> {
let maps = self.uvi.as_ref().ok_or(Error::NoMapsYet)?;
let image = &maps.image;
let map = image.view(0, 0, image.width() / UVI_MAP_COUNT, image.height());
let coords = project(&*map, UVI_MAP_REF_POINTS, position)?;
let stamp = maps.timestamp_base;
sample(image, stamp, UVI_MAP_INTERVAL, UVI_MAP_COUNT, coords)
}
}
impl MapsRefresh for MapsHandle {
fn is_pollen_stale(&self) -> bool {
let maps = self.lock().expect("Maps handle mutex was poisoned");
match &maps.pollen {
Some(pollen_maps) => {
Utc::now().signed_duration_since(pollen_maps.mtime)
> Duration::seconds(POLLEN_MAP_COUNT as i64 * POLLEN_MAP_INTERVAL)
}
None => false,
}
}
fn is_uvi_stale(&self) -> bool {
let maps = self.lock().expect("Maps handle mutex was poisoned");
match &maps.uvi {
Some(uvi_maps) => {
Utc::now().signed_duration_since(uvi_maps.mtime)
> Duration::seconds(UVI_MAP_COUNT as i64 * UVI_MAP_INTERVAL)
}
None => false,
}
}
fn needs_pollen_refresh(&self) -> bool {
let maps = self.lock().expect("Maps handle mutex was poisoned");
match &maps.pollen {
Some(pollen_maps) => {
Utc::now()
.signed_duration_since(pollen_maps.mtime)
.num_seconds()
> POLLEN_INTERVAL
}
None => true,
}
}
fn needs_uvi_refresh(&self) -> bool {
let maps = self.lock().expect("Maps handle mutex was poisoned");
match &maps.uvi {
Some(uvi_maps) => {
Utc::now()
.signed_duration_since(uvi_maps.mtime)
.num_seconds()
> UVI_INTERVAL
}
None => true,
}
}
fn set_pollen(&self, retrieved_maps: Result<RetrievedMaps>) {
if retrieved_maps.is_ok() || self.is_pollen_stale() {
let mut maps = self.lock().expect("Maps handle mutex was poisoned");
maps.pollen = retrieved_maps.ok();
}
}
fn set_uvi(&self, retrieved_maps: Result<RetrievedMaps>) {
if retrieved_maps.is_ok() || self.is_uvi_stale() {
let mut maps = self.lock().expect("Maps handle mutex was poisoned");
maps.uvi = retrieved_maps.ok();
}
}
}
/// A Buienradar map sample.
///
/// This represents a value at a given time.
#[derive(Clone, Debug, PartialEq, Serialize)]
#[serde(crate = "rocket::serde")]
pub(crate) struct Sample {
/// The time(stamp) of the forecast.
#[serde(serialize_with = "ts_seconds::serialize")]
pub(crate) time: DateTime<Utc>,
/// The forecasted score.
///
/// A value in the range `1..=10`.
#[serde(rename(serialize = "value"))]
pub(crate) score: u8,
}
impl Sample {
#[cfg(test)]
pub(crate) fn new(time: DateTime<Utc>, score: u8) -> Self {
Self { time, score }
}
}
/// Builds a scoring histogram for the map key.
fn map_key_histogram() -> MapKeyHistogram {
MAP_KEY
.into_iter()
.fold(HashMap::new(), |mut hm, channels| {
hm.insert(Rgb::from(channels), 0);
hm
})
}
/// Samples the provided maps at the given (map-relative) coordinates and starting timestamp.
/// It assumes the provided coordinates are within bounds of at least one map.
/// The interval is the number of seconds the timestamp is bumped for each map.
fn sample<I: GenericImageView<Pixel = Rgba<u8>>>(
image: &I,
stamp: DateTime<Utc>,
interval: i64,
count: u32,
coords: (u32, u32),
) -> Result<Vec<Sample>> {
let (x, y) = coords;
let width = image.width() / count;
let height = image.height();
if x > width || y > height {
return Err(Error::OutOfBoundCoords(x, y));
}
let max_sample_width = (width - x).min(MAP_SAMPLE_SIZE[0]);
let max_sample_height = (height - y).min(MAP_SAMPLE_SIZE[1]);
let mut samples = Vec::with_capacity(count as usize);
let mut time = stamp;
let mut offset = 0;
while offset < image.width() {
let map = image.view(
x.saturating_sub(MAP_SAMPLE_SIZE[0] / 2) + offset,
y.saturating_sub(MAP_SAMPLE_SIZE[1] / 2),
max_sample_width,
max_sample_height,
);
let histogram = map
.pixels()
.fold(map_key_histogram(), |mut h, (_px, _py, color)| {
h.entry(color.to_rgb()).and_modify(|count| *count += 1);
h
});
let (max_color, &count) = histogram
.iter()
.max_by_key(|(_color, count)| *count)
.expect("Map key is never empty");
if count == 0 {
return Err(Error::NoKnownColorsInSamples);
}
let score = MAP_KEY
.iter()
.position(|&color| &Rgb::from(color) == max_color)
.map(|score| score + 1) // Scores go from 1..=10, not 0..=9!
.expect("Maximum color is always a map key color") as u8;
samples.push(Sample { time, score });
time += Duration::seconds(interval);
offset += width;
}
Ok(samples)
}
/// A retrieved image with some metadata.
#[derive(Debug)]
pub(crate) struct RetrievedMaps {
/// The image data.
pub(crate) image: DynamicImage,
/// The date/time the image was last modified.
pub(crate) mtime: DateTime<Utc>,
/// The starting date/time the image corresponds with.
pub(crate) timestamp_base: DateTime<Utc>,
}
impl RetrievedMaps {
#[cfg(test)]
pub(crate) fn new(image: DynamicImage) -> Self {
let mtime = Utc::now();
let timestamp_base = Utc::now();
Self {
image,
mtime,
timestamp_base,
}
}
}
/// Retrieves an image from the provided URL.
async fn retrieve_image(url: Url) -> Result<RetrievedMaps> {
let response = reqwest::get(url).await?;
let mtime = match response.headers().get(reqwest::header::LAST_MODIFIED) {
Some(mtime_header) => {
let mtime_headr_str = mtime_header.to_str()?;
DateTime::from(DateTime::parse_from_rfc2822(mtime_headr_str)?)
}
None => Utc::now(),
};
let timestamp_base = {
let path = response.url().path();
let (_, filename) = path
.rsplit_once('/')
.ok_or_else(|| Error::InvalidImagePath(path.to_owned()))?;
let (timestamp_str, _) = filename
.split_once("__")
.ok_or_else(|| Error::InvalidImagePath(path.to_owned()))?;
let timestamp = NaiveDateTime::parse_from_str(timestamp_str, "%Y%m%d%H%M")?;
Utc.from_utc_datetime(&timestamp)
};
let bytes = response.bytes().await?;
tokio::task::spawn_blocking(move || {
image::load_from_memory_with_format(&bytes, ImageFormat::Png)
.map(|image| RetrievedMaps {
image,
mtime,
timestamp_base,
})
.map_err(Error::from)
})
.await?
}
/// Retrieves the pollen maps from Buienradar.
///
/// See [`POLLEN_BASE_URL`] for the base URL and [`retrieve_image`] for the retrieval function.
async fn retrieve_pollen_maps() -> Result<RetrievedMaps> {
let timestamp = format!("{}", chrono::Local::now().format("%y%m%d%H%M"));
let mut url = Url::parse(POLLEN_BASE_URL).unwrap();
url.query_pairs_mut().append_pair("timestamp", &timestamp);
println!("🗺️ Refreshing pollen maps from: {}", url);
retrieve_image(url).await
}
/// Retrieves the UV index maps from Buienradar.
///
/// See [`UVI_BASE_URL`] for the base URL and [`retrieve_image`] for the retrieval function.
async fn retrieve_uvi_maps() -> Result<RetrievedMaps> {
let timestamp = format!("{}", chrono::Local::now().format("%y%m%d%H%M"));
let mut url = Url::parse(UVI_BASE_URL).unwrap();
url.query_pairs_mut().append_pair("timestamp", &timestamp);
println!("🗺️ Refreshing UV index maps from: {}", url);
retrieve_image(url).await
}
/// Returns the map for the given instant.
fn map_at(
image: &DynamicImage,
stamp: DateTime<Utc>,
interval: i64,
count: u32,
instant: DateTime<Utc>,
) -> Result<DynamicImage> {
let duration = instant.signed_duration_since(stamp);
let offset = (duration.num_seconds() / interval) as u32;
// Check if out of bounds.
if offset >= count {
return Err(Error::OutOfBoundOffset(offset));
}
let width = image.width() / count;
Ok(image.crop_imm(offset * width, 0, width, image.height()))
}
/// Marks the provided coordinates on the map using a horizontal and vertical line.
fn mark(mut image: DynamicImage, coords: (u32, u32)) -> DynamicImage {
let (x, y) = coords;
for py in 0..image.height() {
image.put_pixel(x, py, Rgba::from([0x00, 0x00, 0x00, 0x70]));
}
for px in 0..image.width() {
image.put_pixel(px, y, Rgba::from([0x00, 0x00, 0x00, 0x70]));
}
image
}
/// Projects the provided geocoded position to a coordinate on a map.
///
/// This uses two reference points and a Mercator projection on the y-coordinates of those points
/// to calculate how the map scales with respect to the provided position.
fn project<I: GenericImageView>(
image: &I,
ref_points: [(Position, (u32, u32)); 2],
pos: Position,
) -> Result<(u32, u32)> {
// Get the data from the reference points.
let (ref1, (ref1_y, ref1_x)) = ref_points[0];
let (ref2, (ref2_y, ref2_x)) = ref_points[1];
// For the x-coordinate, use a linear scale.
let scale_x = ((ref2_x - ref1_x) as f64) / (ref2.lon_as_rad() - ref1.lon_as_rad());
let x = ((pos.lon_as_rad() - ref1.lon_as_rad()) * scale_x + ref1_x as f64).round() as u32;
// For the y-coordinate, use a Mercator-projected scale.
let mercator_y = |lat: f64| (lat / 2.0 + PI / 4.0).tan().ln();
let ref1_merc_y = mercator_y(ref1.lat_as_rad());
let ref2_merc_y = mercator_y(ref2.lat_as_rad());
let scale_y = ((ref1_y - ref2_y) as f64) / (ref2_merc_y - ref1_merc_y);
let y = ((ref2_merc_y - mercator_y(pos.lat_as_rad())) * scale_y + ref2_y as f64).round() as u32;
if image.in_bounds(x, y) {
Ok((x, y))
} else {
Err(Error::OutOfBoundCoords(x, y))
}
}
/// Returns the data of a map with a crosshair drawn on it for the given position.
///
/// The map that is used is determined by the provided metric.
pub(crate) async fn mark_map(
position: Position,
metric: Metric,
maps_handle: &MapsHandle,
) -> crate::Result<Vec<u8>> {
use std::io::Cursor;
let maps_handle = Arc::clone(maps_handle);
tokio::task::spawn_blocking(move || {
let maps = maps_handle.lock().expect("Maps handle lock was poisoned");
let image = match metric {
Metric::Pollen => maps.pollen_mark(position),
Metric::UVI => maps.uvi_mark(position),
_ => return Err(crate::Error::UnsupportedMetric(metric)),
}?;
drop(maps);
// Encode the image as PNG image data.
let mut image_data = Cursor::new(Vec::new());
match image.write_to(&mut image_data, ImageFormat::Png) {
Ok(()) => Ok(image_data.into_inner()),
Err(err) => Err(crate::Error::from(Error::from(err))),
}
})
.await
.map_err(Error::from)?
}
/// Runs a loop that keeps refreshing the maps when necessary.
///
/// Use [`MapsRefresh`] trait methods on `maps_handle` to check whether each maps type needs to be
/// refreshed and uses its retrieval function to update it if necessary.
pub(crate) async fn run(maps_handle: MapsHandle) {
loop {
println!("🕔 Refreshing the maps (if necessary)...");
if maps_handle.needs_pollen_refresh() {
let retrieved_maps = retrieve_pollen_maps().await;
if let Err(e) = retrieved_maps.as_ref() {
eprintln!("💥 Encountered error during pollen maps refresh: {}", e);
}
maps_handle.set_pollen(retrieved_maps);
}
if maps_handle.needs_uvi_refresh() {
let retrieved_maps = retrieve_uvi_maps().await;
if let Err(e) = retrieved_maps.as_ref() {
eprintln!("💥 Encountered error during UVI maps refresh: {}", e);
}
maps_handle.set_uvi(retrieved_maps);
}
sleep(REFRESH_INTERVAL).await;
}
}

117
src/position.rs Normal file
View File

@ -0,0 +1,117 @@
//! Positions in the geographic coordinate system.
//!
//! This module contains everything related to geographic coordinate system functionality.
use std::f64::consts::PI;
use std::hash::Hash;
use cached::proc_macro::cached;
use geocoding::{Forward, Openstreetmap, Point};
use rocket::tokio;
use crate::{Error, Result};
/// A (geocoded) position.
///
/// This is used for measuring and communication positions directly on the Earth as latitude and
/// longitude.
///
/// # Position equivalence and hashing
///
/// For caching purposes we need to check equivalence between two positions. If the positions match
/// up to the 5th decimal, we consider them the same (see [`Position::lat_as_i32`] and
/// [`Position::lon_as_i32`]).
#[derive(Clone, Copy, Debug, Default)]
pub(crate) struct Position {
/// The latitude of the position.
pub(crate) lat: f64,
/// The longitude of the position.
pub(crate) lon: f64,
}
impl Position {
/// Creates a new (geocoded) position.
pub(crate) const fn new(lat: f64, lon: f64) -> Self {
Self { lat, lon }
}
/// Returns the latitude as an integer.
///
/// This is achieved by multiplying it by `10_000` and rounding it. Thus, this gives a
/// precision of 5 decimals.
fn lat_as_i32(&self) -> i32 {
(self.lat * 10_000.0).round() as i32
}
/// Returns the longitude as an integer.
///
/// This is achieved by multiplying it by `10_000` and rounding it. Thus, this gives a
/// precision of 5 decimals.
fn lon_as_i32(&self) -> i32 {
(self.lon * 10_000.0).round() as i32
}
/// Returns the latitude in radians.
pub(crate) fn lat_as_rad(&self) -> f64 {
self.lat * PI / 180.0
}
/// Returns the longitude in radians.
pub(crate) fn lon_as_rad(&self) -> f64 {
self.lon * PI / 180.0
}
/// Returns the latitude as a string with the given precision.
pub(crate) fn lat_as_str(&self, precision: usize) -> String {
format!("{:.*}", precision, self.lat)
}
/// Returns the longitude as a string with the given precision.
pub(crate) fn lon_as_str(&self, precision: usize) -> String {
format!("{:.*}", precision, self.lon)
}
}
impl From<&Point<f64>> for Position {
fn from(point: &Point<f64>) -> Self {
// The `geocoding` API always returns (longitude, latitude) as (x, y).
Position::new(point.y(), point.x())
}
}
impl Hash for Position {
fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
// Floats cannot be hashed. Use the 5-decimal precision integer representation of the
// coordinates instead.
self.lat_as_i32().hash(state);
self.lon_as_i32().hash(state);
}
}
impl PartialEq for Position {
fn eq(&self, other: &Self) -> bool {
self.lat_as_i32() == other.lat_as_i32() && self.lon_as_i32() == other.lon_as_i32()
}
}
impl Eq for Position {}
/// Resolves the geocoded position for a given address.
///
/// If the result is [`Ok`], it will be cached.
/// Note that only the 100 least recently used addresses will be cached.
#[cached(size = 100, result = true)]
pub(crate) async fn resolve_address(address: String) -> Result<Position> {
println!("🌍 Geocoding the position of the address: {}", address);
tokio::task::spawn_blocking(move || {
let osm = Openstreetmap::new();
let points: Vec<Point<f64>> = osm.forward(&address)?;
points
.first()
.ok_or(Error::NoPositionFound)
.map(Position::from)
})
.await?
}

7
src/providers.rs Normal file
View File

@ -0,0 +1,7 @@
//! All supported metric data providers.
//!
//! Data is either provided via a direct (JSON) API or via looking up values on maps.
pub(crate) mod buienradar;
pub(crate) mod combined;
pub(crate) mod luchtmeetnet;

225
src/providers/buienradar.rs Normal file
View File

@ -0,0 +1,225 @@
//! The Buienradar data provider.
//!
//! For more information about Buienradar, see: <https://www.buienradar.nl/overbuienradar/contact>
//! and <https://www.buienradar.nl/overbuienradar/gratis-weerdata>.
use cached::proc_macro::cached;
use chrono::serde::ts_seconds;
use chrono::{DateTime, Datelike, Duration, NaiveTime, ParseError, TimeZone, Utc};
use chrono_tz::Europe;
use csv::ReaderBuilder;
use reqwest::Url;
use rocket::serde::{Deserialize, Serialize};
use crate::maps::MapsHandle;
use crate::position::Position;
use crate::{Error, Metric, Result};
/// The base URL for the Buienradar API.
const BUIENRADAR_BASE_URL: &str = "https://gpsgadget.buienradar.nl/data/raintext";
/// The Buienradar pollen/UV index map sample.
pub(crate) type Sample = crate::maps::Sample;
/// A row in the precipitation text output.
///
/// This is an intermediate type used to represent rows of the output.
#[derive(Debug, Deserialize)]
#[serde(crate = "rocket::serde")]
struct Row {
/// The precipitation value in the range `0..=255`.
value: u16,
/// The time in the `HH:MM` format.
time: String,
}
/// The Buienradar API precipitation data item.
#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)]
#[serde(crate = "rocket::serde", try_from = "Row")]
pub(crate) struct Item {
/// The time(stamp) of the forecast.
#[serde(serialize_with = "ts_seconds::serialize")]
pub(crate) time: DateTime<Utc>,
/// The forecasted value.
///
/// Its unit is mm/h.
pub(crate) value: f32,
}
impl TryFrom<Row> for Item {
type Error = ParseError;
fn try_from(row: Row) -> Result<Self, Self::Error> {
let time = parse_time(&row.time)?;
let value = convert_value(row.value);
Ok(Item { time, value })
}
}
/// Parses a time string to date/time in the UTC time zone.
///
/// The provided time has the format `HH:MM` and is considered to be in the Europe/Amsterdam
/// time zone.
fn parse_time(t: &str) -> Result<DateTime<Utc>, ParseError> {
// First, get the current date in the Europe/Amsterdam time zone.
let today = Utc::now().with_timezone(&Europe::Amsterdam).date_naive();
// Then, parse the time and interpret it relative to "today".
let ntime = NaiveTime::parse_from_str(t, "%H:%M")?;
let ndtime = today.and_time(ntime);
// Finally, interpret the naive date/time in the Europe/Amsterdam time zone and convert it to
// the UTC time zone.
let ldtime = Europe::Amsterdam.from_local_datetime(&ndtime).unwrap();
let dtime = ldtime.with_timezone(&Utc);
Ok(dtime)
}
/// Converts a precipitation value into an precipitation intensity value in mm/h.
///
/// For the conversion formula, see: <https://www.buienradar.nl/overbuienradar/gratis-weerdata>.
fn convert_value(v: u16) -> f32 {
let base: f32 = 10.0;
let value = base.powf((v as f32 - 109.0) / 32.0);
(value * 10.0).round() / 10.0
}
/// Fix the timestamps of the items either before or after the day boundary.
///
/// If in the Europe/Amsterdam time zone it is still before 0:00, all timestamps after 0:00 need to
/// be bumped up with a day. If it is already after 0:00, all timestamps before 0:00 need to be
/// bumped back with a day.
// TODO: If something in Sinoptik needs unit tests, it is this!
fn fix_items_day_boundary(items: Vec<Item>) -> Vec<Item> {
let now = Utc::now().with_timezone(&Europe::Amsterdam);
// Use noon on the same day as "now" as a comparison moment.
let noon = Europe::Amsterdam
.with_ymd_and_hms(now.year(), now.month(), now.day(), 12, 0, 0)
.single()
.expect("Invalid date: input date is invalid or not unambiguous");
if now < noon {
// It is still before noon, so bump timestamps after noon a day back.
items
.into_iter()
.map(|mut item| {
if item.time > noon {
item.time -= Duration::days(1)
}
item
})
.collect()
} else {
// It is already after noon, so bump the timestamps before noon a day forward.
items
.into_iter()
.map(|mut item| {
if item.time < noon {
item.time += Duration::days(1)
}
item
})
.collect()
}
}
/// Retrieves the Buienradar forecasted precipitation items for the provided position.
///
/// If the result is [`Ok`] it will be cached for 5 minutes for the the given position.
#[cached(time = 300, result = true)]
async fn get_precipitation(position: Position) -> Result<Vec<Item>> {
let mut url = Url::parse(BUIENRADAR_BASE_URL).unwrap();
url.query_pairs_mut()
.append_pair("lat", &position.lat_as_str(2))
.append_pair("lon", &position.lon_as_str(2));
println!("▶️ Retrieving Buienradar data from: {url}");
let response = reqwest::get(url).await?;
let output = response.error_for_status()?.text().await?;
let mut rdr = ReaderBuilder::new()
.has_headers(false)
.delimiter(b'|')
.from_reader(output.as_bytes());
let items: Vec<Item> = rdr.deserialize().collect::<Result<_, _>>()?;
// Check if the first item stamp is (timewise) later than the last item stamp.
// In this case `parse_time` interpreted e.g. 23:00 and later 0:30 in the same day and some
// time stamps need to be fixed.
if items
.first()
.zip(items.last())
.map(|(it1, it2)| it1.time > it2.time)
== Some(true)
{
Ok(fix_items_day_boundary(items))
} else {
Ok(items)
}
}
/// Retrieves the Buienradar forecasted pollen samples for the provided position.
///
/// If the result is [`Ok`] if will be cached for 1 hour for the given position.
#[cached(
time = 3_600,
key = "Position",
convert = r#"{ position }"#,
result = true
)]
async fn get_pollen(position: Position, maps_handle: &MapsHandle) -> Result<Vec<Sample>> {
maps_handle
.lock()
.expect("Maps handle mutex was poisoned")
.pollen_samples(position)
.map_err(Into::into)
}
/// Retrieves the Buienradar forecasted UV index samples for the provided position.
///
/// If the result is [`Ok`] if will be cached for 1 day for the given position.
#[cached(
time = 86_400,
key = "Position",
convert = r#"{ position }"#,
result = true
)]
async fn get_uvi(position: Position, maps_handle: &MapsHandle) -> Result<Vec<Sample>> {
maps_handle
.lock()
.expect("Maps handle mutex was poisoned")
.uvi_samples(position)
.map_err(Into::into)
}
/// Retrieves the Buienradar forecasted map samples for the provided position.
///
/// It only supports the following metric:
/// * [`Metric::Pollen`]
/// * [`Metric::UVI`]
pub(crate) async fn get_samples(
position: Position,
metric: Metric,
maps_handle: &MapsHandle,
) -> Result<Vec<Sample>> {
match metric {
Metric::Pollen => get_pollen(position, maps_handle).await,
Metric::UVI => get_uvi(position, maps_handle).await,
_ => Err(Error::UnsupportedMetric(metric)),
}
}
/// Retrieves the Buienradar forecasted items for the provided position.
///
/// It only supports the following metric:
/// * [`Metric::Precipitation`]
///
pub(crate) async fn get_items(position: Position, metric: Metric) -> Result<Vec<Item>> {
match metric {
Metric::Precipitation => get_precipitation(position).await,
_ => Err(Error::UnsupportedMetric(metric)),
}
}

269
src/providers/combined.rs Normal file
View File

@ -0,0 +1,269 @@
//! The combined data provider.
//!
//! This combines and collates data using the other providers.
use cached::proc_macro::cached;
use chrono::serde::ts_seconds;
use chrono::{DateTime, Utc};
use rocket::serde::Serialize;
pub(crate) use super::buienradar::{self, Sample as BuienradarSample};
pub(crate) use super::luchtmeetnet::{self, Item as LuchtmeetnetItem};
use crate::maps::MapsHandle;
use crate::position::Position;
use crate::{Error, Metric};
/// The possible merge errors that can occur.
#[allow(clippy::enum_variant_names)]
#[derive(Debug, thiserror::Error, PartialEq)]
pub(crate) enum MergeError {
/// No AQI item found.
#[error("No AQI item found")]
NoAqiItemFound,
/// No pollen item found.
#[error("No pollen item found")]
NoPollenItemFound,
/// No AQI item found within 30 minutes of first pollen item.
#[error("No AQI item found within 30 minutes of first pollen item")]
NoCloseAqiItemFound,
/// No pollen item found within 30 minutes of first AQI item.
#[error("No pollen item found within 30 minutes of first AQI item")]
NoClosePollenItemFound,
}
/// The combined data item.
#[derive(Clone, Debug, PartialEq, Serialize)]
#[serde(crate = "rocket::serde")]
pub(crate) struct Item {
/// The time(stamp) of the forecast.
#[serde(serialize_with = "ts_seconds::serialize")]
time: DateTime<Utc>,
/// The forecasted value.
value: f32,
}
impl Item {
#[cfg(test)]
pub(crate) fn new(time: DateTime<Utc>, value: f32) -> Self {
Self { time, value }
}
}
/// Merges pollen samples and AQI items into combined items.
///
/// The merging drops items from either the pollen samples or from the AQI items if they are not
/// stamped within half an hour of the first item of the latest starting series, thus lining them
/// before they are combined.
fn merge(
pollen_samples: Vec<BuienradarSample>,
aqi_items: Vec<LuchtmeetnetItem>,
) -> Result<Vec<Item>, MergeError> {
let mut pollen_samples = pollen_samples;
let mut aqi_items = aqi_items;
// Only retain samples/items that have timestamps that are at least an hour ago.
let now = Utc::now();
pollen_samples.retain(|smp| smp.time.signed_duration_since(now).num_seconds() > -3600);
aqi_items.retain(|item| item.time.signed_duration_since(now).num_seconds() > -3600);
// Align the iterators based on the (hourly) timestamps!
let pollen_first_time = pollen_samples
.first()
.ok_or(MergeError::NoPollenItemFound)?
.time;
let aqi_first_time = aqi_items.first().ok_or(MergeError::NoAqiItemFound)?.time;
if pollen_first_time < aqi_first_time {
// Drain one or more pollen samples to line up.
let idx = pollen_samples
.iter()
.position(|smp| {
smp.time
.signed_duration_since(aqi_first_time)
.num_seconds()
.abs()
< 1800
})
.ok_or(MergeError::NoCloseAqiItemFound)?;
pollen_samples.drain(..idx);
} else {
// Drain one or more AQI items to line up.
let idx = aqi_items
.iter()
.position(|item| {
item.time
.signed_duration_since(pollen_first_time)
.num_seconds()
.abs()
< 1800
})
.ok_or(MergeError::NoClosePollenItemFound)?;
aqi_items.drain(..idx);
}
// Combine the samples with items by taking the maximum of pollen sample score and AQI item
// value.
let items = pollen_samples
.into_iter()
.zip(aqi_items)
.map(|(pollen_sample, aqi_item)| {
let time = pollen_sample.time;
let value = (pollen_sample.score as f32).max(aqi_item.value);
Item { time, value }
})
.collect();
Ok(items)
}
/// Retrieves the combined forecasted items for the provided position and metric.
///
/// It supports the following metric:
/// * [`Metric::PAQI`]
#[cached(
time = 1800,
key = "(Position, Metric)",
convert = r#"{ (position, metric) }"#,
result = true
)]
pub(crate) async fn get(
position: Position,
metric: Metric,
maps_handle: &MapsHandle,
) -> Result<Vec<Item>, Error> {
if metric != Metric::PAQI {
return Err(Error::UnsupportedMetric(metric));
};
let pollen_items = buienradar::get_samples(position, Metric::Pollen, maps_handle).await?;
let aqi_items = luchtmeetnet::get(position, Metric::AQI).await?;
let items = merge(pollen_items, aqi_items)?;
Ok(items)
}
#[cfg(test)]
mod tests {
use chrono::{Duration, Timelike};
use super::*;
#[test]
fn merge() {
let t_now = Utc::now()
.with_second(0)
.unwrap()
.with_nanosecond(0)
.unwrap();
let t_m2 = t_now.checked_sub_signed(Duration::days(1)).unwrap();
let t_m1 = t_now.checked_sub_signed(Duration::hours(2)).unwrap();
let t_0 = t_now.checked_add_signed(Duration::minutes(12)).unwrap();
let t_1 = t_now.checked_add_signed(Duration::minutes(72)).unwrap();
let t_2 = t_now.checked_add_signed(Duration::minutes(132)).unwrap();
let pollen_samples = Vec::from([
BuienradarSample::new(t_m2, 4),
BuienradarSample::new(t_m1, 5),
BuienradarSample::new(t_0, 1),
BuienradarSample::new(t_1, 3),
BuienradarSample::new(t_2, 2),
]);
let aqi_items = Vec::from([
LuchtmeetnetItem::new(t_m2, 4.0),
LuchtmeetnetItem::new(t_m1, 5.0),
LuchtmeetnetItem::new(t_0, 1.1),
LuchtmeetnetItem::new(t_1, 2.9),
LuchtmeetnetItem::new(t_2, 2.4),
]);
// Perform a normal merge.
let merged = super::merge(pollen_samples.clone(), aqi_items.clone());
assert!(merged.is_ok());
let paqi = merged.unwrap();
assert_eq!(
paqi,
Vec::from([
Item::new(t_0, 1.1),
Item::new(t_1, 3.0),
Item::new(t_2, 2.4),
])
);
// The pollen samples are shifted, i.e. one hour in the future.
let shifted_pollen_samples = pollen_samples[2..]
.iter()
.cloned()
.map(|mut item| {
item.time = item.time.checked_add_signed(Duration::hours(1)).unwrap();
item
})
.collect::<Vec<_>>();
let merged = super::merge(shifted_pollen_samples, aqi_items.clone());
assert!(merged.is_ok());
let paqi = merged.unwrap();
assert_eq!(paqi, Vec::from([Item::new(t_1, 2.9), Item::new(t_2, 3.0)]));
// The AQI items are shifted, i.e. one hour in the future.
let shifted_aqi_items = aqi_items[2..]
.iter()
.cloned()
.map(|mut item| {
item.time = item.time.checked_add_signed(Duration::hours(1)).unwrap();
item
})
.collect::<Vec<_>>();
let merged = super::merge(pollen_samples.clone(), shifted_aqi_items);
assert!(merged.is_ok());
let paqi = merged.unwrap();
assert_eq!(paqi, Vec::from([Item::new(t_1, 3.0), Item::new(t_2, 2.9)]));
// The maximum sample/item should not be later then the interval the PAQI items cover.
let merged = super::merge(pollen_samples[..3].to_vec(), aqi_items.clone());
assert!(merged.is_ok());
let paqi = merged.unwrap();
assert_eq!(paqi, Vec::from([Item::new(t_0, 1.1)]));
let merged = super::merge(pollen_samples.clone(), aqi_items[..3].to_vec());
assert!(merged.is_ok());
let paqi = merged.unwrap();
assert_eq!(paqi, Vec::from([Item::new(t_0, 1.1)]));
// Merging fails because the samples/items are too far (6 hours) apart.
let shifted_aqi_items = aqi_items
.iter()
.cloned()
.map(|mut item| {
item.time = item.time.checked_add_signed(Duration::hours(6)).unwrap();
item
})
.collect::<Vec<_>>();
let merged = super::merge(pollen_samples.clone(), shifted_aqi_items);
assert_eq!(merged, Err(MergeError::NoCloseAqiItemFound));
let shifted_pollen_samples = pollen_samples
.iter()
.cloned()
.map(|mut item| {
item.time = item.time.checked_add_signed(Duration::hours(6)).unwrap();
item
})
.collect::<Vec<_>>();
let merged = super::merge(shifted_pollen_samples, aqi_items.clone());
assert_eq!(merged, Err(MergeError::NoClosePollenItemFound));
// The pollen samples list is empty, or everything is too old.
let merged = super::merge(Vec::new(), aqi_items.clone());
assert_eq!(merged, Err(MergeError::NoPollenItemFound));
let merged = super::merge(pollen_samples[0..2].to_vec(), aqi_items.clone());
assert_eq!(merged, Err(MergeError::NoPollenItemFound));
// The AQI items list is empty, or everything is too old.
let merged = super::merge(pollen_samples.clone(), Vec::new());
assert_eq!(merged, Err(MergeError::NoAqiItemFound));
let merged = super::merge(pollen_samples, aqi_items[0..2].to_vec());
assert_eq!(merged, Err(MergeError::NoAqiItemFound));
}
}

View File

@ -0,0 +1,85 @@
//! The Luchtmeetnet open data provider.
//!
//! For more information about Luchtmeetnet, see: <https://www.luchtmeetnet.nl/contact>.
use cached::proc_macro::cached;
use chrono::serde::ts_seconds;
use chrono::{DateTime, Duration, Utc};
use reqwest::Url;
use rocket::serde::{Deserialize, Serialize};
use crate::position::Position;
use crate::{Error, Metric, Result};
/// The base URL for the Luchtmeetnet API.
const LUCHTMEETNET_BASE_URL: &str = "https://api.luchtmeetnet.nl/open_api/concentrations";
/// The Luchtmeetnet API data container.
///
/// This is only used temporarily during deserialization.
#[derive(Debug, Deserialize)]
#[serde(crate = "rocket::serde")]
struct Container {
data: Vec<Item>,
}
/// The Luchtmeetnet API data item.
#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)]
#[serde(crate = "rocket::serde")]
pub(crate) struct Item {
/// The time(stamp) of the forecast.
#[serde(
rename(deserialize = "timestamp_measured"),
serialize_with = "ts_seconds::serialize"
)]
pub(crate) time: DateTime<Utc>,
/// The forecasted value.
///
/// The unit depends on the selected [metric](Metric).
pub(crate) value: f32,
}
impl Item {
#[cfg(test)]
pub(crate) fn new(time: DateTime<Utc>, value: f32) -> Self {
Self { time, value }
}
}
/// Retrieves the Luchtmeetnet forecasted items for the provided position and metric.
///
/// It supports the following metrics:
/// * [`Metric::AQI`]
/// * [`Metric::NO2`]
/// * [`Metric::O3`]
/// * [`Metric::PM10`]
#[cached(time = 1800, result = true)]
pub(crate) async fn get(position: Position, metric: Metric) -> Result<Vec<Item>> {
let formula = match metric {
Metric::AQI => "lki",
Metric::NO2 => "no2",
Metric::O3 => "o3",
Metric::PM10 => "pm10",
_ => return Err(Error::UnsupportedMetric(metric)),
};
let mut url = Url::parse(LUCHTMEETNET_BASE_URL).unwrap();
url.query_pairs_mut()
.append_pair("formula", formula)
.append_pair("latitude", &position.lat_as_str(5))
.append_pair("longitude", &position.lon_as_str(5));
println!("▶️ Retrieving Luchtmeetnet data from: {url}");
let response = reqwest::get(url).await?;
let root: Container = response.error_for_status()?.json().await?;
// Filter items that are older than one hour before now. They seem to occur sometimes?
let too_old = Utc::now() - Duration::hours(1);
let items = root
.data
.into_iter()
.filter(|item| item.time > too_old)
.collect();
Ok(items)
}