Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSM Offline using HTTP server concept #389

Open
na7q opened this issue Nov 1, 2024 · 18 comments
Open

OSM Offline using HTTP server concept #389

na7q opened this issue Nov 1, 2024 · 18 comments

Comments

@na7q
Copy link
Contributor

na7q commented Nov 1, 2024

Not fully understanding the magsforge dilemma for offline maps. I have an idea that could be a valid workaround for the time being.

Hosting tiles (png) using a simple http server that hosts the images locally on the device. I was able to make this method work, however I ran into a problem with mapsforge forcing port 80 for an http server. Even with explicitly coding 8080 for example.

Where does the mapsforgev3 jar come from? Is there an easy to find source to compile it? Seems it holds the key to the port 80 problem.

I could also be wrong on mapsforge being the problem here.

3:42.463 28741-28769 OsmTileDownloader org.aprsdroid.app D Constructed URL: http://127.0.0.1:8080/11/320/727.png 2024-10-31 18:43:43.182 28741-28769 OsmTileDownloader org.aprsdroid.app D Server response: OK, tile fetched successfully 2024-10-31 18:43:43.207 28741-28769 TileDownloader org.aprsdroid.app E null java.net.ConnectException: Failed to connect to /127.0.0.1:80 at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:147) at com.android.okhttp.internal.io.RealConnection.connect(RealConnection.java:116) at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:186) at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:128) at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:97) at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:289) at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:232) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:465) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:411) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:248) at org.mapsforge.v3.android.maps.mapgenerator.tiledownloader.TileDownloader.executeJob(TileDownloader.java:65) at org.mapsforge.v3.android.maps.mapgenerator.MapWorker.doWork(MapWorker.java:73) at org.mapsforge.v3.android.maps.PausableThread.run(PausableThread.java:94)

@penguin359
Copy link
Contributor

The exact version of Mapsforge that APRSdroid is based on, 0.3.0, is from 2012 and seems to be from an era where they used a Subversion repository hosted on Google Code, and not the GitHub repo they use now, though I suspect that the commits were imported into it. Interestingly enough, the oldest tag they have in Git is actually 0.3.1 and the Subversion repository with the 0.3.0 tag is no more. That Jar file, being 12 years old, is probably no longer on any Maven repos, otherwise, it would be simple to pull down. What I can tell you is that the two Maven components/artifacts are is the following:

  • org.mapsforge:mapsforge-map:0.3.0
  • org.mapsforge:mapsforge-map-reader:0.3.0

An approximation of them can probably be grabbed from here:

@penguin359
Copy link
Contributor

Ultimately, a key issue is that we are just stuck on such an old version of Mapsforge and updating requires a bit of code rework to support the newer API. I think the 0.5.x branch, in particular, they changed the underlying file format so the newer map tiles aren't compatible with the older version of the library. I might be wrong on the exact versions.

What I'm thinking about to become a little more familiar with the code is to just try to bump it to the next minor release. It looks like 0.4.1 is the oldest version of Mapsforge currently available on the Maven Central repository so the following lines added to build.gradle can pull it in:

implementation 'org.mapsforge:mapsforge-map:0.4.1'
implementation 'org.mapsforge:mapsforge-map-reader:0.4.1'

That's still quite old and it's probably best to just jump to 0.21.0 from the repo.

@penguin359
Copy link
Contributor

penguin359 commented Nov 3, 2024

Looking through their repo, it looks like they switched from Subversion to Git somewhere between 0.3.0 and 0.3.1 and they didn't bother importing any of the history from Subversion. I can't do a git diff or log to follow the changes made after 0.3.0. I assumed that 0.3.1 should be mostly compatible, but it looks like they made significant changes to it and it is not source compatible at all.

I'm currently trying to compile APRSdroid against mapsforge 0.3.1 as a stepping stone, but even that is requiring some changes. Mostly, it's that they dropped the .v3 component from the package path and moved a few other classes around. However, some classes like ItemizedOverlay and MapGeneratorFactory appear to have been completely dropped so this will take a little while. I can see why he hasn't bothered to update to newer releases.

@na7q
Copy link
Contributor Author

na7q commented Nov 3, 2024

For most people, the map is truly the most important part of this all. While I don't use the maps much at all, it would be nice to have back, but with the newer better offline maps.

I have other ideas, but since we can download OSM tiles as PNG a lot easier than reworking this code. I figured this route would be doable. At least until I hit that hardcoded port problem.

@na7q
Copy link
Contributor Author

na7q commented Nov 3, 2024

https://github.com/cgeo/mapsforge-old
I think this is the original here.

@na7q
Copy link
Contributor Author

na7q commented Nov 4, 2024

That was it! I modified it, recompiled. Modified APRSdroid, and it works! Offline maps baby!

@na7q
Copy link
Contributor Author

na7q commented Nov 4, 2024

Initial commit enabling offline usage for png tile server. 127.0.0.1:8080
na7q@0a47d18

Easiest way to obtain the maps is to pull the APRSIS32 OSMtiles folder. Put it on your devices storage. Run any HTTP server on port 8080 to that folder of tiles. Enable offline map mode, and that's it. It'll use those tiles.

@gunnardave
Copy link

Thank you for looking into this.

For what it's worth I use maps from OpenAndroidMaps (in Mapsforge V3, no longer hosted on the site, I archived all the U.S. maps before they disappeared) with my APRSdroid installation.

https://www.openandromaps.org/en/downloads/usa_en

To save storage I share the Mapsforge maps with other apps such as Backcountry Navigator and Oruxmaps, which is my main Android navigation app. Orux is a handy little map app since it can display Mapsforge, PDF (for example I use National Park, Forest Service and BLM topos, among other GeoPDF), Garmin IMG and many other types.

I'm not sure the details on how but it can handle both V3 and V5 Mapsforge. I have not tried V4 maps.

https://www.oruxmaps.com/cs/en/

Again, thank you! To me having an offline map is a critical feature of APRSdroid.

David
AC0VH

@na7q
Copy link
Contributor Author

na7q commented Nov 4, 2024

Users like you are why I wanted to get it working again. I've come up with an easy method to obtain maps. I have a big bug to fix in my code related to digipeating that causes the app to stop when an invalid packet is heard. Otherwise it's working far better than the original offline maps ever did (I think).

@na7q
Copy link
Contributor Author

na7q commented Nov 5, 2024

Some minimal instructions. More to come.
https://na7q.com/aprsdroid-osm/

@na7q
Copy link
Contributor Author

na7q commented Nov 28, 2024

Just an FYI. I have added support for MBTILES! The website is now out of date. I will update it soon.
If rooted you can even use mbtiles directly from Gaia maps and other apps that use mbtiles. More info to come.

@ge0rg
Copy link
Owner

ge0rg commented Dec 27, 2024

Thanks for the interesting contribution! I have an internal branch using the org.mapsforge:vtm-android library, but it's lacking the symbol / trace rendering yet. The goal is to restructure the map rendering code in a fashion where Google Maps, VTM and old-school mapsforge can be used equally, with live switching in the same APK. For VTM, the goal is also to enumerate all the map files in a given folder and to allow selecting them from a drop-down, similarly how it used to be in c:geo. Keeping the mapsforge library is not a problem then, however it's not hidpi-capable, so all the map labels are really tiny. Maybe we need to increase the mapsforge library regardless.

@na7q
Copy link
Contributor Author

na7q commented Dec 27, 2024

Thanks for the interesting contribution! I have an internal branch using the org.mapsforge:vtm-android library, but it's lacking the symbol / trace rendering yet. The goal is to restructure the map rendering code in a fashion where Google Maps, VTM and old-school mapsforge can be used equally, with live switching in the same APK. For VTM, the goal is also to enumerate all the map files in a given folder and to allow selecting them from a drop-down, similarly how it used to be in c:geo. Keeping the mapsforge library is not a problem then, however it's not hidpi-capable, so all the map labels are really tiny. Maybe we need to increase the mapsforge library regardless.

Glad to see you over here. Not only have I managed this, but in the last couple days I created an internal tile server within APRSdroid that runs at start. So termux isn't needed, or any external apps. It is working perfectly with mbtiles mapping, loading speeds are extremely fast. Here's the current branch of that.
https://github.com/na7q/aprsdroid/tree/OSM-SERVER

It's an interesting and weird concept, but has been proven successful. Plus it gives a wide variety of options for maps!
It has one dilemma that being it require "MANAGE_EXTERNAL_STORAGE" permissions. I'm not great with Android apps, so getting this granted is problematic. Using adb shell I can workaround this.
"adb shell appops set --uid org.aprsdroid.app MANAGE_EXTERNAL_STORAGE allow"

Furthermore, I need to find a way to add more online mapping options within the map viewer, but I can't figure out why the map viewer doesn't properly switch tile servers when selecting it from the list.

As you can see, I'm really winging my way through.

@na7q
Copy link
Contributor Author

na7q commented Dec 27, 2024

I would definitely love to see what you have with that map work. I don't think I could help much there, but I'd participate where possible.

@ge0rg
Copy link
Owner

ge0rg commented Dec 27, 2024

I've uploaded my WIP code to https://github.com/ge0rg/aprsdroid/tree/vtm with an added VTMMapAct class.

@na7q
Copy link
Contributor Author

na7q commented Dec 27, 2024

By the way, here's the offline maps server running.
https://www.youtube.com/shorts/n1uenCZCPcI

Thanks for the link, I will check out that branch!

@kd2var
Copy link

kd2var commented Dec 28, 2024

By the way, here's the offline maps server running. https://www.youtube.com/shorts/n1uenCZCPcI

Thanks for the link, I will check out that branch!

What is the mapsforge apk that you posted on your website today?

@na7q
Copy link
Contributor Author

na7q commented Dec 29, 2024

By the way, here's the offline maps server running. https://www.youtube.com/shorts/n1uenCZCPcI
Thanks for the link, I will check out that branch!

What is the mapsforge apk that you posted on your website today?

That was using a built in webserver in aprsdroid. I've moved on to a direct method that mapsforge can directly read the mbtiles. Linked below, but requires added permissions which can only be given with adb at this time. Email me for more details. It's a WIP.
The one on my website uses an external server, as described in the instructions.

https://na7q.com/wp-content/uploads/2024/12/aprsdroid-release-noapi-mapsforge-mbtiles.apk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants