All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- GitHub actions config
- Feature parsing in
AugmentedDiffSource
usesvectorpipe.util.RobustFeature
to tolerate bad geometries in the stream #148. - Receive GPG key while publishing artifacts #138
Pipeline#finalize(vectorTiles, zoom)
method to receive the final RDD of generated vector tiles for a zoom levelPipeline.Output
mixin trait that overridesfinalize
with default implementation usingsaveVectorTiles(vectorTiles, zoom, pipeline.baseOutputURI)
VectorPipe.Options
to support for any square layout level (not just from ZoomedLayoutScheme)Pipeline#baseOutputURI
moved toPipeline.Output#baseOutputURI
- Updated Geotrellis dependency to 3.5.1
- Improve robustness of functions in
vectorpipe.sources.ChangesetSource
- Catch 403 S3Exceptions when checking minutely diffs in AugmentedDiffSource
- Catch proper AWS SDK v2 NoSuchKeyException when checking minutely diffs in AugmentedDiffSource
- AugmentedDiffSource failed to properly decode from JSON sources
- MicroBatchReader null pointer exception when reading baseURI from DataSourceOptions
vectorpipe.examples
: VectorPipe examples moved from https://github.com/azavea/osmesaVectorPipe.defaultSparkSessionWithJTS
method to construct a VectorPipe tailoredSparkSession
. Users with more complicated use cases will still want to manually construct their own session.
This is the first release to depend on GeoTrellis 3.0.
- Streaming sources now fallback to the current remote sequence if no database checkpoint or option can be found
- Depend on Spark 2.4.4
- Depend on GeoTrellis 3.1.0
useCaching
option to VectorPipe.Options allows for persisting to disk. Helps avoid repeated computations.- Functions for converting sequence numbers to timestamps and back for both
changeset replications and augmented diff replications. See
ChangesetSource
andAugmentedDiffSource
invectorpipe.sources
.
- Improved empty geometry handling in UDFs
- RELEASING.md - Instructions for releasing new versions of this project
- Support for semicolon-delimited tag values in UDFs, e.g.
shop=bakery;dairy
- Support for
nds
in augmented diff GeoJSON (matchingosm-replication-streams@^0.7.0
output) - "Uninteresting" tags are dropped when processing OSM inputs; this will result in fewer point features being generated (as those nodes previously had tags applied).
- Sync with [email protected] for determining area-ness of a way.
- Fetch gzipped augmented diff JSON (produced by overpass-diff-publisher)
- Preserve the last-known coordinates of deleted nodes
- Better handling of falsy boolean values in tag UDFs
- Adds
riverbank
,stream_end
,dam
,weir
,waterfall
, andpressurised
to the list of waterway features - Populates
nds
andmembers
for deleted elements from the previous version
- Resolve commons-io deprecation warnings
- Convert coordinates to Doubles (expected by VP internals) when pre-processing
- Mark all logger vals and some UDF vals as @transient lazy to avoid Spark serialization issues
- Properly strip leading and trailing slashes from S3 URIs when exporting vector tiles