Skip to content

Commit

Permalink
Merge pull request #31 from chswx/release/2.1
Browse files Browse the repository at this point in the history
Ingestor 2.1
  • Loading branch information
jaredwsmith authored Mar 28, 2021
2 parents 3096e7c + 0f54d14 commit 226cd7d
Show file tree
Hide file tree
Showing 63 changed files with 49,864 additions and 698 deletions.
5 changes: 5 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
DB_NAME="chswx"
DB_SERVER="localhost"

LOG_LEVEL="Logger::DEBUG"
LOG_OUTPUT="php://stdout"
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,7 @@ conf/*
lib/*
*.swp
vendor/*
.idea
.vscode
.phpunit.result.cache
.env
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
language: php
php:
- 5.6
- 7.3
install:
- composer install
25 changes: 17 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,27 @@ A series of scripts to ingest and store NWS watch/warning/advisory products via

## New in 2.0

All responsibilities for outputting the results of parsing out products from LDM will be placed on the [Alerter](http://github.com/chswx/alerter) going forward. Thus, a lot of what's new in 2.0 has been more code _deletion_ than anything else.
Version 2.0, instead of handling the entire lifecycle of a request from receiving it from LDM to sending it out over Twitter, will just dump everything in a pub-sub-aware database (likely RethinkDB but don't quote me on this just yet) and let other worker processes figure it out. This should improve performance, scalability, and redundancy quite nicely. We store things in JSON; this makes it easy to send the data to virtually anywhere, including directly over a socket into a Web browser. (Wink
All responsibilities for outputting the results of parsing out products from LDM will be placed on the [Alerter](http://github.com/chswx/alerter) going forward. Thus, a lot of what's new in 2.0 has been more code _deletion_ than anything else.
Version 2.0, instead of handling the entire lifecycle of a request from receiving it from LDM to sending it out over Twitter, will just dump everything in a pub-sub-aware database (currently RethinkDB) and let other worker processes figure it out. This should improve performance, scalability, and redundancy quite nicely. We store things in JSON; this makes it easy to send the data to virtually anywhere, including directly over a socket into a Web browser. (Wink
wink.)

## What’s done

- VTEC (Valid Time Extent Code) awareness
- JSON output
- Impact-based warnings
- Storm-based warnings
- Pub/sub architecture based on RethinkDB

## What’s in the works

- Support for watch probabilities (WWUS40)
- Impact-based warnings (TORNADO…HAIL…WIND tags)
- Moving to a totally pub/sub architecture
- Support for parsing Special Weather Statements/Significant Weather Advisories (#13)
- Support for watch probabilities (WWUS40) (#14)
- Parsing and geocoding Mesoscale Convective/Precip Discussions from SPC and WPC (#15)
- SPC outlooks (#16)
- LSRs (#17)
- Tropical advisories (#18)
- Climate products (#23)
- Awareness of conditions via METAR ingest
- Awareness of forecast updates

Expand All @@ -30,16 +37,18 @@ wink.)

## See it in action

Follow [@chswx on Twitter](http://twitter.com/chswx) during inclement weather in Charleston, SC to see examples of the bot’s output.
The current version of the LDM ingestor works behind the scenes.

To see its [predecessor](https://github.com/chswx/ldm-ingest-legacy) in action, follow [@chswx on Twitter](http://twitter.com/chswx) during inclement weather in Charleston, SC to see examples of the bot’s output.

## Questions?

File issues on GitHub against this project or contact me on Twitter: [@jaredwsmith](http://twitter.com/jaredwsmith)

## Licensing, warranty, etc.

Licensing TBD. Code is pretty lousy and should not be used for life and death applications. No warranty implied. Use at your own risk.
Licensing TBD. Not for use in life or death situations; code is getting better but the level of rigor is below what I would expect for mission-critical things.

## Many Thanks

Many thanks to @blairblends, @edarc and @UpdraftNetworks for infrastructure and seeding the initial code for this project.
Many thanks to @blairblends, @edarc and @chswx\LDMIngest for infrastructure and seeding the initial code for this project.
26 changes: 20 additions & 6 deletions composer.json
Original file line number Diff line number Diff line change
@@ -1,13 +1,27 @@
{
"name": "chswx/ldm-ingest",
"description": "Receives products from Unidata's LDM and parses them into JSON, which can then be reused in many, many wonderful ways.",
"require": {
"danielmewes/php-rql": "^2.2"
"danielmewes/php-rql": "^2.2",
"monolog/monolog": "^2.0",
"vlucas/phpdotenv": "^3.6"
},
"require-dev": {
"phpunit/phpunit": "^9"
},
"autoload": {
"psr-4": {
"UpdraftNetworks\\":"src/",
"UpdraftNetworks\\Ingestor\\Tests\\":"tests/",
"UpdraftNetworks\\Parser\\Tests\\":"tests/",
"":"src/"
"chswx\\LDMIngest\\": "src/",
"chswx\\LDMIngest\\Ingestor\\Tests\\": "tests/",
"chswx\\LDMIngest\\Parser\\Tests\\": "tests/",
"": "src/"
}
},
"license": "GPLv2",
"authors": [
{
"name": "Jared Smith",
"email": "[email protected]"
}
}
]
}
Loading

0 comments on commit 226cd7d

Please sign in to comment.