Bump 9.6-3.1 to GEOS 3.9.1 #233

TomPohys · 2021-04-15T16:04:06Z

Hi all,

I would like to ask if there is a plan to update GEOS to 3.9.1 (https://github.com/libgeos/geos/releases)?

As I thought wrongly, update HASH for master still leads to GEOS 3.7.1 (c580901) and not upgrade the GEOS version to 3.9.1.

I am trying to resolve a GEOS bump in OpenMapTiles.

Thanks for comment

The text was updated successfully, but these errors were encountered:

nyurik · 2021-04-15T17:03:46Z

I think it would be great to upgrade all 3.1 tags to the latest official release of geo libraries, not just 12-master and 13-master, while keeping postgres & postgis to the designated versions. Older Postgis versions might not be not as important for the upgrade. An alternative would be to create another set of all tags, e.g. 9.6-3.1-3.9.1, but that seem too much, and adds another dimension to the variants (might be ok though)

ImreSamu · 2021-04-16T05:52:15Z

my personal opinion - for brainstorming
( I am still thinking 🤔 )

@TomPohys:

I would like to ask if there is a plan to update GEOS to 3.9.1

IMHO: this is mostly an upstream ( Debian packaging) issue.
As I know

for the current Debian version - no Geos update expected
for the next Debian (bullseye) - now: geos 3.9.0 ( and not 3.9.1 )

see the related e-mail thread

IMHO: no easy way to "Bump 9.6-3.1 to GEOS 3.9.1"

We need to build the geos 3.9.1
and we need to rebuild the PostGIS lib
- search for "Conditional Compilation" #if POSTGIS_GEOS_VERSION < 39
  - https://github.com/postgis/postgis/search?q=%22%23if+POSTGIS_GEOS_VERSION%22
and we need to preserve the "compatibility" for the other users

@nyurik:

I think it would be great to upgrade all 3.1 tags to the latest official release of geo libraries, not just 12-master and 13-master,
while keeping postgres & postgis to the designated versions.

my personal opinion/suggestions:

1.) adding more "build time" customizations to the "master Dockerfile" (base image, versions)

adding ARG instructions
- to create ~ any combinations for the "local" customized build.
keeping the current compatibility - with the "-master"

so in theory you need to run this script:

# ... draft example - for your use cases  ...
rm -rf docker-postgis
git clone --quiet --depth 1 https://github.com/postgis/docker-postgis.git
cd docker-postgis/13-master

# customized local build 
docker build --pull \
  \
  --build-arg BASE_PG_IMAGE=postgres:9.6 \
  \
  --build-arg SFCGAL_VERSION=v1.3.9 \
  --build-arg SFCGAL_GIT_HASH=823db7a318b8841f8296e80036ef993ddf19ebf5 \
  --build-arg PROJ_VERSION=8.0.0 \
  --build-arg PROJ_GIT_HASH=12e7a4395784d2dfb45cc0b719d37cd7e534fe91 \
  --build-arg GEOS_VERSION=3.9.1 \
  --build-arg GEOS_GIT_HASH=1b2f4bd3e19001c37846687c765595aa8c28b90f \
  --build-arg GDAL_VERSION=v3.2.2 \
  --build-arg GDAL_GIT_HASH=a33784291d19015217ea2604988e53d448e14a07 \
  --build-arg POSTGIS_VERSION=3.1.1 \
  --build-arg POSTGIS_GIT_HASH=aaf4c79a4906f11727fe6b9bccb12051a52acfe3 \
  \
  -t postgis:pg9.6-pgis3.1.1-geos3.9.1-proj8.0.0-gdal3.2.2  .

# final step:
# -->  Push a new image to your registry ( https://hub.docker.com/u/openmaptiles )

IMHO: I am deeply conservative - what we add to the postgis docker-hub

the PG 9.6 Final release = "November 11, 2021"
the debian11(bullseye) - is near.
and lot of use cases .. it is easy to add;
- on the other hand hard to remove / change / support
  - so need a deep discussion

IMHO: Later: if the -master script is more mature

We can add a ~13-latest; 14-latest version only for the last 2 Postgres version;

2.) adding Debian:bullseye based PostGIS images as early as we can

Now I am testing the Debian "11/testing/bullseye" based - postgis3.1+Geos3.9.0 version
- Debian 11: expected this calendar year, it is now in a "Hard Freeze" status;
- IMHO: usable ( --> I am using in my pet projects )
- my test postgis:13-3.1-bullseye
  - docker size ( ~ 567MB )
  - new LLVM(for a posgres JIT!)/Geos3.9/Proj7.2/Gdal3.2/...
maybe we can add this version in the next months; with an extra -bullseye tag
- The biggest problem - no upstream postgres-bullseye versions yet
  - we need to ask.
  - or temporary we need to build ourselves.
For a long time - there will be the same problem with a locked Geos3.9/Proj7.2/Gdal3.2 version.

ImreSamu · 2021-04-17T04:21:33Z

has a "known" side effect, the 3.9+ only functions not available.

see this e-mail question:

nyurik · 2021-04-18T00:19:02Z

Thanks @ImreSamu for the in-depth reply! I wonder if we can utilize something like INCLUDE+ dockerfile command (extension). This works with the recent dockers - I tried it with 20.10, Linux containers only. To build, one must use DOCKER_BUILDKIT=1 docker build ... to enable buildkit mode. The rest will be done automatically by the docker build service when it sees the # syntax = edrevo/dockerfile-plus comment in the dockerfile.

Essentially, the idea would be to make current templates files into "common" dockerfiles that can be included from specific versions, but also to make those templates usable directly by downstream projects. I don't know if all such features would be supported, needs some testing.

nyurik · 2021-04-18T00:37:49Z

See also a relevant question at edrevo/dockerfile-plus#6

ImreSamu · 2021-04-18T11:29:44Z

@nyurik

Essentially, the idea would be to make current templates files into "common" dockerfiles
that can be included from specific versions, but also to make those templates usable directly by downstream projects.
I don't know if all such features would be supported, needs some testing.

Please: give more info; What is the problem you want to solve?

a current practical example - with code fragments.

As I understand - you want to split the Dockerfile.master.template; How? and Which part is important for you?

nyurik · 2021-04-18T11:57:08Z

@ImreSamu above you gave a draft example of custom build with many build args. I wonder if it would make sense to do it purely in dockerfile rather than with extra bash/makefile commands. It might be more trouble than its worth though, just an idea.

ImreSamu · 2021-04-18T12:58:30Z

I wonder if it would make sense to do it purely in dockerfile rather than with extra bash/makefile commands.
It might be more trouble than its worth though, just an idea.

I just want to add a few ARG to the Dockerfile.master.template

With keeping the current compatibility
With no change in the current workflow. ( ~ keeping the current "bash/makefile commands" )
With no other extra complexity.

IMHO: Changing the basic Dockerfile syntax .. adds extra complexity; and need a Discussions

ImreSamu · 2021-04-18T19:45:46Z

the good news: Debian Bullseye expected in the next months ( May/June/... )

"Finding a tentative bullseye release date" : https://lists.debian.org/debian-release/2021/04/msg00187.html
"The Release Team believes that the state of bullseye is pretty good."
"... We propose to aim for a release date in May."

ImreSamu · 2021-04-23T15:21:59Z

Status:
"Debian Installer Bullseye RC 1 release" ( Fri, 23 Apr 2021 07:21:43 +0200 )
https://lists.debian.org/debian-boot/2021/04/msg00192.html

13-3.1-bullseye version :

select postgis_full_version();
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                            postgis_full_version                                                                            |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| POSTGIS="3.1.1 aaf4c79" [EXTENSION] PGSQL="130" GEOS="3.9.0-CAPI-1.16.2" PROJ="7.2.1" LIBXML="2.9.10" LIBJSON="0.15" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" TOPOLOGY |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
(1 row)

ImreSamu · 2021-07-05T02:28:36Z

Debian bullseye expected on 2021-07-31 http://debiantracker.herokuapp.com/
My proposal WIP : WIP: build-arg api v1 proposal #247 ( This is only a proposal; "merge" is not guaranteed )

TomPohys · 2021-07-05T09:18:04Z

Great! Thank you for pushing the next steps!

ImreSamu · 2021-09-27T20:59:43Z

I think it would be great to upgrade all 3.1 tags to the latest official release of geo libraries,

The #257 has been merged; so now

all PostGIS3.1 images are Bullseye based now. ( ---> GEOS="3.9.0" )
And docker pull postgis/postgis:14rc1-3.1 is available for testing.
- https://github.com/postgis/docker-postgis/blob/1cf67643bb955faadf0ec24e70aeb94ee262e912/14rc1-3.1/Dockerfile

postgis/postgis:14rc1-3.1

psql (14rc1 (Debian 14~rc1-1.pgdg110+1))
Type "help" for help.

postgres=# select postgis_full_version();
                                                                            postgis_full_version                                                                            
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="3.1.3 008d2db" [EXTENSION] PGSQL="140" GEOS="3.9.0-CAPI-1.16.2" PROJ="7.2.1" LIBXML="2.9.10" LIBJSON="0.15" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" TOPOLOGY
(1 row)

postgres=# select version() ;
                                                            version                                                             
--------------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 14rc1 (Debian 14~rc1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
(1 row)

postgis/postgis:13-3.1

psql (13.4 (Debian 13.4-1.pgdg110+1))
Type "help" for help.

postgres=# select postgis_full_version();
                                                                            postgis_full_version                                                                            
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="3.1.4 ded6c34" [EXTENSION] PGSQL="130" GEOS="3.9.0-CAPI-1.16.2" PROJ="7.2.1" LIBXML="2.9.10" LIBJSON="0.15" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" TOPOLOGY
(1 row)

postgres=# select version() ;
                                                           version                                                           
-----------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 13.4 (Debian 13.4-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
(1 row)

nyurik · 2021-10-03T01:44:55Z

@ImreSamu hi, any updates on this? I think the most stable way for OpenMapTiles would be to use your #247 with custom ARG params. This way this repo will maintain HOW to build a docker image with all dependencies, and OMT repo will specify WHAT exact versions of each component to use. This way we can have PG14+GEOS3.9.1+... as the latest PG image.

One other thought - is there a way to factor out Postgis+deps building into a separate script that Dockerfile calls during the build phase? For the heavy-duty production systems we currently install PG directly on GCP VM, and install the needed code using a startup script. It would be great if PG installation script could be centrally maintained, and users would just use it for their setup?

Thanks for all your hard work in this repo!

ImreSamu · 2021-10-06T20:52:19Z

@nyurik

with custom ARG params. ... any updates on this?

Still working on this.

I want to add default package support
the new SFCGAL requires CGAL >= 5.3 ; and bullseye has CGAL 5.2
etc;

This way we can have PG14+GEOS3.9.1+... as the latest PG image.

If It is urgent :
Temporary you can use the current https://github.com/postgis/docker-postgis/blob/master/Dockerfile.master.template
just you have to replace the parameters (%%PROJ_GIT_HASH%%,%%GEOS_GIT_HASH%%,%%GDAL_GIT_HASH%%,%%POSTGIS_GIT_HASH%%, etc )
with a simple script. ( sed? https://github.com/postgis/docker-postgis/blob/master/update.sh#L95 )

e357e0a7381a8c2f36c62375380acbf800da697d --> %%POSTGIS_GIT_HASH%% -> and you have: 3.2.0alpha1

And later you can upgrade to the new API.

... we currently install PG directly on GCP VM, ...
It would be great if PG installation script could be centrally maintained, and users would just use it for their setup?

I don't have a quick answer.
so please post your request to the: https://github.com/postgis/docker-postgis/discussions

IMHO:

If full consistency is important, I think you should convert the Dockerfile to a shell script - with a tool. ( probably need some research )

ImreSamu · 2023-11-17T21:52:50Z

Since this issue is quite old and Postgis 9.6 is no longer supported, I am closing this issue.
If you have any new information related to this, please let me know, and I will reopen it.
Alternatively, feel free to create a new issue.

TomPohys mentioned this issue Apr 17, 2021

Bump GEOS 3.9.1 openmaptiles/openmaptiles-tools#345

Merged

TomPohys mentioned this issue Apr 23, 2021

building aggregation fails with GEOSUnaryUnion: TopologyException openmaptiles/openmaptiles#1058

Closed

nyurik mentioned this issue May 10, 2021

Migrate to a custom PostGIS docker openmaptiles/openmaptiles-tools#350

Open

ImreSamu mentioned this issue Jul 4, 2021

WIP: build-arg api v1 proposal #247

Draft

4 tasks

ImreSamu closed this as completed Nov 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bump 9.6-3.1 to GEOS 3.9.1 #233

Bump 9.6-3.1 to GEOS 3.9.1 #233

TomPohys commented Apr 15, 2021

nyurik commented Apr 15, 2021

ImreSamu commented Apr 16, 2021

ImreSamu commented Apr 17, 2021

nyurik commented Apr 18, 2021

nyurik commented Apr 18, 2021

ImreSamu commented Apr 18, 2021

nyurik commented Apr 18, 2021

ImreSamu commented Apr 18, 2021

ImreSamu commented Apr 18, 2021

ImreSamu commented Apr 23, 2021

ImreSamu commented Jul 5, 2021

TomPohys commented Jul 5, 2021

ImreSamu commented Sep 27, 2021

nyurik commented Oct 3, 2021

ImreSamu commented Oct 6, 2021

ImreSamu commented Nov 17, 2023

Bump 9.6-3.1 to GEOS 3.9.1 #233

Bump 9.6-3.1 to GEOS 3.9.1 #233

Comments

TomPohys commented Apr 15, 2021

nyurik commented Apr 15, 2021

ImreSamu commented Apr 16, 2021

1.) adding more "build time" customizations to the "master Dockerfile" (base image, versions)

2.) adding Debian:bullseye based PostGIS images as early as we can

ImreSamu commented Apr 17, 2021

nyurik commented Apr 18, 2021

nyurik commented Apr 18, 2021

ImreSamu commented Apr 18, 2021

nyurik commented Apr 18, 2021

ImreSamu commented Apr 18, 2021

ImreSamu commented Apr 18, 2021

ImreSamu commented Apr 23, 2021

13-3.1-bullseye version :

ImreSamu commented Jul 5, 2021

TomPohys commented Jul 5, 2021

ImreSamu commented Sep 27, 2021

postgis/postgis:14rc1-3.1

postgis/postgis:13-3.1

nyurik commented Oct 3, 2021

ImreSamu commented Oct 6, 2021

ImreSamu commented Nov 17, 2023