Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NOAA-APT #72

Open
leonardpitzu opened this issue Jun 2, 2024 · 19 comments
Open

NOAA-APT #72

leonardpitzu opened this issue Jun 2, 2024 · 19 comments
Labels
enhancement New feature or request

Comments

@leonardpitzu
Copy link

How about the NOAA-APT downloaded images using sat dump? I dod not find anything in the documentation and the folder structure is kind of flat...
folder name is "2024-06-01_19-16_noaa_dsb_137.77 MHz" and inside there-s a bunch of png's and the dataset json.

@ImDroided
Copy link

I am literally trying to add my RaspberryNOAA images right now. I have all the new ini files made but I cant get the images to load.

added to NOAA and METEOR the config.ini and it shows up on the webpage menu

[paths]
GOES16 = "/srv/live_output/GOES16"
GOES18 = "/srv/live_output/GOES18"
NOAA = "/srv/images"
METEOR = "/srv/images"

[categories]
abi = abi.ini
abi2 = abi2.ini
noaa = noaa.ini
meteor = meteor.ini
meso = meso.ini
meso2 = meso2.ini
nws = nws.ini
emwin = emwin.ini

Next I made noaa.ini with the following

[_category_]
title = "NOAA APT Imagary"
icon = fas fa-satellite

[mcir_latest]
path = "{NOAA}"
title = "NOAA MCIR Latest"
fast = true
mode = beginu
filter = "MCIR"
color = #003241

And meteor.ini

[_category_]
title = "Meteor HRPT Imagary"
icon = fas fa-satellite

[fdfc_16]
path = "{METEOR}"
title = "Meteor Latest"
fast = false
mode = endz
filter = "equidistant_321_composite"
color = #003241

I have tried to rename the image to match one of the mode listings and still nothing loads I have tried every mode and same thing. Not sure where I am going wrong. My image files are saved in the following format

NOAA-15-20240528-005747-MCIR.jpg
METEOR-M2-4-20240602-203249-equidistant_321_composite.jpg

Once I get one of the images to load I will add the rest.

@leonardpitzu
Copy link
Author

I also got here but no images are displayed...

@ImDroided
Copy link

@creinemann posted this on FB earlier but I am still getting no love from it. Still says no images found.

447777644_10161214635113280_5223489356707999210_n

This is my NOAA ini file I have no idea whats up here. I am not sure if its my file names or not.

[_category_]
title = "NOAA APT Imagary"
icon = fas fa-satellite

[mcir_latest]
path = "{NOAA}/"
title = "NOAA MCIR Latest"
fast = true
mode = end
filter = NOAA-15-
color = #003241

@creinemann
Copy link

creinemann commented Jun 3, 2024 via email

@ImDroided
Copy link

If you read my post, you will see that my method involves modifying the regex in goes vitality and scripts renaming NOAA and meteor imagery, this is outside of the normal usage of what the author of Goes Vitality has created and requires some substantial editing of code.

On Mon, Jun 3, 2024, 5:36 PM ImDroided @.> wrote: @creinemann https://github.com/creinemann posted this on FB earlier but I am still getting no love from it. Still says no images found. 447777644_10161214635113280_5223489356707999210_n.jpg (view on web) https://github.com/JVital2013/vitality-goes/assets/13988713/91b2fd49-a42b-4486-8c7f-876255502ecc This is my NOAA ini file I have no idea whats up here. I am not sure if its my file names or not. [category] title = "NOAA APT Imagary" icon = fas fa-satellite [mcir_latest] path = "{NOAA}/" title = "NOAA MCIR Latest" fast = true mode = end filter = NOAA-15- color = #003241 — Reply to this email directly, view it on GitHub <#72 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALGT3Q74477D6HT3VZ6O5Q3ZFTVWVAVCNFSM6AAAAABIVI3URSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBWGI2DCOBSG4 . You are receiving this because you were mentioned.Message ID: @.>

Your right I completely missed the see more part of the post. I will dive into this further later and figure out how I want to tackle it.

@ImDroided
Copy link

Ok so I have added noaa_apt to datahandler.php

switch($config['categories'][$_GET['id']]['data'][$_GET['subid']]['mode'])
		{
			case "satdump_geo":
				$regex = "/(\\\\|\/)" . $DateTime->format('Y-m-d_H-i-s') . "(\\\\|\/)[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}\..{3}$/i";
				break;
			case "noaa_apt":
				$regex = "/(\\\\|\/)" . $DateTime->format('Ymd-His') . "[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*\..{3}$/i";
				break;
			case "begin":
				$regex = "/(\\\\|\/)" . $DateTime->format('YmdHis') . "[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*\..{3}$/i";
				break;
			case "beginu":
				$regex = "/(\\\\|\/)" . $DateTime->format('Ymd_His') . "[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*\..{3}$/i";
				break;
			case "beginz":
				$regex = "/(\\\\|\/)" . $DateTime->format('Ymd\THis\Z') . "[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*\..{3}$/i";
				break;
			case "emwin":
				$regex = "/_" . $DateTime->format('YmdHis') . "_[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*\..{3}$/i";
				break;
			case "xrit":
				$regex = "/{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*" . $DateTime->format('YmdHi') . "\..{3}$/i";
				break;
			case "end":
				$regex = "/{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*" . $DateTime->format('YmdHis') . "\..{3}$/i";
				break;
			case "endu":
				$regex = "/{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*" . $DateTime->format('Ymd_His') . "\..{3}$/i";
				break;
			case "endz":
				$regex = "/{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*" . $DateTime->format('Ymd\THis\Z') . "\..{3}$/i";
				break;
			default: die(); break;
		}

and also functions.php

function getFormatByMode($mode, $filter)
{
	switch($mode)
	{
		case "satdump_geo":
			$regex = "/(\\\\|\/)(?<date>[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}-[0-9]{2}-[0-9]{2})(\\\\|\/)[^\\\\\/]*{$filter}\..{3}$/i";
			$dateFormat = "Y-m-d_H-i-s";
			break;
		case "noaa_apt":
			$regex = "/(\\\\|\/)(?<date>[0-9]{8}-[0-9]{6})[^\\\\\/]*{$filter}[^\\\\\/]*\..{3}$/i";
			$dateFormat = "Ymd-His";
			break;
		case "begin":
			$regex = "/(\\\\|\/)(?<date>[0-9]{14})[^\\\\\/]*{$filter}[^\\\\\/]*\..{3}$/i";
			$dateFormat = "YmdHis";
			break;
		case "beginu":
			$regex = "/(\\\\|\/)(?<date>[0-9]{8}_[0-9]{6})[^\\\\\/]*{$filter}[^\\\\\/]*\..{3}$/i";
			$dateFormat = "Ymd_His";
			break;
		case "beginz":
			$regex = "/(\\\\|\/)(?<date>[0-9]{8}T[0-9]{6}Z)[^\\\\\/]*{$filter}[^\\\\\/]*\..{3}$/i";
			$dateFormat = "Ymd\THis\Z";
			break;
		case "xrit":
			$regex = "/{$filter}[^\\\\\/]*(?<date>[0-9]{12})\..{3}$/i";
			$dateFormat = "YmdHi";
			break;
		case "emwin":
			$regex = "/_(?<date>[0-9]{14})_[^\\\\\/]*{$filter}[^\\\\\/]*\..{3}$/i";
			$dateFormat = "YmdHis";
			break;
		case "end":
			$regex = "/{$filter}[^\\\\\/]*(?<date>[0-9]{14})\..{3}$/i";
			$dateFormat = "YmdHis";
			break;
		case "endu":
			$regex = "/{$filter}[^\\\\\/]*(?<date>[0-9]{8}_[0-9]{6})\..{3}$/i";
			$dateFormat = "Ymd_His";
			break;
		case "endz":
			$regex = "/{$filter}[^\\\\\/]*(?<date>[0-9]{8}T[0-9]{6}Z)\..{3}$/i";
			$dateFormat = "Ymd\THis\Z";
			break;
		default: die("Invalid server config: $mode is not a valid file parser mode!"); break;
	}

But I am getting

The server returned bad data: Invalid server config: noaa_apt is not a valid file parser mode!

What am I missing to include this in the regex modes? I've searched all the files and I am missing it. I don't see anything further that I have to add.

@ImDroided
Copy link

I dont know why but after redoing both edits its added now.

@ImDroided
Copy link

ImDroided commented Jun 4, 2024

I also got here but no images are displayed...

Ok so I got it sort of working... It loads the page and all the image data but its a broken image.

image
image

This is what got the files to be seen from the default output of RaspberryNOAA with the file names in the post above. If you post the satdump file structure and filename I can try to help with that.

case "noaa_apt":
		$regex = "/(\\\\|\/)" . $DateTime->format('Ymd-His') . "[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}[^\\\\\/]*\..{3}$/i";
		break;

But again I have broken images but all the time stamps are correct. Ill get this figured out hopefully tonight

@ImDroided
Copy link

I am stuck here. I have no idea who the images are not displaying. I've looked through the code and I see nothing that stands out on what's wrong here. My timestamps are correct and image count is correct and also changes if I change the filter so I know its seeing the files. The image permissions are the same as all my goes images. I have even tried changing the filter to pull the png images in the directory with the same result. @JVital2013 what am I missing here?

@creinemann
Copy link

creinemann commented Jun 4, 2024

Just a quick glance but, are you missing a $ in front of the "filter", which would look for your filter keyword?

case "noaa_apt":
		$regex = "/(\\\\|\/)" . $DateTime->format('Ymd-His') . "[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['$filter']}[^\\\\\/]*\..{3}$/i";
		break;

@ImDroided
Copy link

ImDroided commented Jun 4, 2024

Just a quick glance but, are you missing a $ in front of the "filter", which would look for your filter keyword?

case "noaa_apt":
		$regex = "/(\\\\|\/)" . $DateTime->format('Ymd-His') . "[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['$filter']}[^\\\\\/]*\..{3}$/i";
		break;

That made no difference. its seeing the files and even displaying the correct timestamps and image counts but it is not producing the actual image. if I try to open image in new tab I get this page

http://website.com/dataHandler.php?type=data&id=noaa&subid=mcir_latest&timestamp=1717519527

but its a blank page.... And I just realized it is a timestamp issue. its linking to a timestamp that does not exist. the timestamp for the current image is 20240604-164527 I have tried this timestamp and also without the - and it is loading nothing.

EDIT: It seems to be praising the timestamps correctly because the text under the image is correct but it is not sending the correct link for the image from datahandler.php

@ImDroided
Copy link

So the url dataHandler.php?type=data&id=noaa&subid=mcir_latest produces the latest image so its defiantly a timestamp issue. Hopefully @JVital2013 can tell me what I am messing up here.

@JVital2013
Copy link
Owner

JVital2013 commented Jun 7, 2024

Looks like I'm late to the party!

I've been thinking about implementing something like this, but the mode will probably be called satdump_leo instead of noaa_apt because it will work for all SatDump LEO captures.

The regex passed around here doesn't work quite the way I'd want, and some of them like I'm seeing with yours @ImDroided simply won't work. Here's some working code (tested and works without any renaming of received files from SatDump):

dataHandler.php

...
		switch($config['categories'][$_GET['id']]['data'][$_GET['subid']]['mode'])
		{
			case "satdump_geo":
				$regex = "/(\\\\|\/)" . $DateTime->format('Y-m-d_H-i-s') . "(\\\\|\/)[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}\..{3}$/i";
				break;
			case "satdump_leo":
				$regex = "/(\\\\|\/)" . $DateTime->format('Y-m-d_H-i') . ".*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}\..{3}$/i";
				break;
...

functions.php

...
function getFormatByMode($mode, $filter)
{
	switch($mode)
	{
		case "satdump_geo":
			$regex = "/(\\\\|\/)(?<date>[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}-[0-9]{2}-[0-9]{2})(\\\\|\/)[^\\\\\/]*{$filter}\..{3}$/i";
			$dateFormat = "Y-m-d_H-i-s";
			break;
		case "satdump_leo":
			$regex = "/(\\\\|\/)(?<date>[0-9]{4}-[0-9]{2}-[0-9]{2}_[0-9]{2}-[0-9]{2}).*{$filter}\..{3}$/i";
			$dateFormat = "Y-m-d_H-i";
			break;
...

Example .ini section

[meteor_321]
path="/home/jamie/live_output"
title = "METEOR MSU-MR 321 (Filled & Corrected)"
filter = "MSU-MR \(Filled\)[\\\/]msu_mr_rgb_321_corrected"
mode = "satdump_leo"

[meteor_221]
path="/home/jamie/live_output"
title = "METEOR MSU-MR 221 (Filled & Corrected)"
filter = "MSU-MR \(Filled\)[\\\/]msu_mr_rgb_221_corrected"
mode = "satdump_leo"

[allnoaa_mcir]
path="/home/jamie/live_output"
title = "NOAA (ALL) - MCIR"
filter = "avhrr_3_rgb_MCIR"
mode = "satdump_leo"

[noaa19_mcir]
path="/home/jamie/live_output"
title = "NOAA 19 - APT MCIR"
filter = "137.1 MHz[\\\/]avhrr_3_rgb_MCIR"
mode = "satdump_leo"

In this mode, the filter setting must match the end of the file path, but it can also include the path name before it as shown in these examples. I'm not sure if I will do it this way in the end, but it works for now!

image

@JVital2013 JVital2013 added the enhancement New feature or request label Jun 7, 2024
@ImDroided
Copy link

ImDroided commented Jun 7, 2024

@JVital2013 Thanks for that. I got this working last night somewhat with this

DataHandler.php

                        case "raspberrynoaa":
				$regex = "/{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['filter']}-" . $DateTime->format('Ymd-His') . "-[^\\\\\/]*\..{3}$/i";
				break;
			case "raspberrynoaa_latest":
				$regex = "/-" . $DateTime->format('Ymd-His') . "[^\\\\\/]*{$config['categories'][$_GET['id']]['data'][$_GET['subid']]['$filter']}\..{3}$/i";
				break;

functions.php

                case "raspberrynoaa":
			$regex = "/{$filter}-(?<date>[0-9]{8}-[0-9]{6})[^\\\\\/]*[^\\\\\/]*\..{3}$/i";
			$dateFormat = "Ymd-His";
			break;
		case "raspberrynoaa_latest":
			$regex = "/-(?<date>[0-9]{8}-[0-9]{6})[^\\\\\/]-*{$filter}[^\\\\\/]*\..{3}$/i";
			$dateFormat = "Ymd-His";
			break;

noaa.ini

[_category_]
title = "NOAA APT Imagary"
icon = fas fa-satellite

[noaa-15]
path = "{NOAA}/"
title = "NOAA 15 Latest"
fast = true
mode = raspberrynoaa
filter = NOAA-15
color = #003241

[noaa-18]
path = "{NOAA}/"
title = "NOAA 18 Latest"
fast = true
mode = raspberrynoaa
filter = NOAA-18
color = #003241

[noaa-19]
path = "{NOAA}/"
title = "NOAA 19 Latest"
fast = true
mode = raspberrynoaa
filter = NOAA-19
color = #003241

[mcir_latest]
path = "{NOAA}/"
title = "NOAA MCIR Latest"
fast = true
mode = raspberrynoaa_latest
filter = MCIR-precip
color = #003241

and got this last night

Screenshot 2024-06-07 012502

but it now looks like this

image

This is the file structure for RaspberryNOAA

image

I'm close lol but I'm determined. I'm not sure why I am getting non MCIR images at all for the latest image as it should be the last precip image but I get others.

How are you guys scheduling passes with satdump? I have only ever scheduled satdump with the GUI but it would always crash after a day or 3. I have no issue switching over to satdump completely but there are a few guys on FB wanting to incorporate RaspberryNOAA imagery

And I just noticed I spelled imagery wrong in all my ini's lmao. I have to stop doing these things at 2am

EDIT:..... I added your code and I switched RaspberryNOAA to use satdump there is a setting in the config file so we will see how this goes in 30 minutes if it gets a good pass then hopefully it will show up on the site.

@ImDroided
Copy link

@JVital2013 Meteor is not loading with your example it is saying no images found.

This is working for all the NOAA sats

[allnoaa_mcir]
path="{NOAA}"
title = "NOAA (ALL) - MCIR"
filter = "avhrr_3_rgb_MCIR"
mode = "satdump_leo"

[noaa15_mcir]
path="{NOAA}"
title = "NOAA 15 - APT MCIR"
filter = "137.62 MHz[\\\/]avhrr_3_rgb_MCIR"
mode = "satdump_leo"

[noaa18_mcir]
path="{NOAA}"
title = "NOAA 18 - APT MCIR"
filter = "137.912 MHz[\\\/]avhrr_3_rgb_MCIR"
mode = "satdump_leo"

[noaa19_mcir]
path="{NOAA}"
title = "NOAA 19 - APT MCIR"
filter = "137.1 MHz[\\\/]avhrr_3_rgb_MCIR"
mode = "satdump_leo"

@ImDroided
Copy link

I should add these

image

config.ini

[paths]
GOES16 = "/srv/live_output/GOES16"
GOES18 = "/srv/live_output/GOES18"
NOAA = "/srv/live_output/LEO"
METEOR = "/srv/live_output/LEO"

I only modified the path for METEOR

[meteor_321]
path="{METEOR}"
title = "METEOR MSU-MR 321 (Filled & Corrected)"
filter = "MSU-MR \(Filled\)[\\\/]msu_mr_rgb_321_corrected"
mode = "satdump_leo"

[meteor_221]
path="{METEOR}"
title = "METEOR MSU-MR 221 (Filled & Corrected)"
filter = "MSU-MR \(Filled\)[\\\/]msu_mr_rgb_221_corrected"
mode = "satdump_leo"

@JC167shearer
Copy link

Hi all, I'm trying to implement this also, but am having trouble with the filter section of my noaa.ini. If I comment out the filter section entirely, I get all the images in my specified directory (demonstrating that everything except image selection is working?). I cannot find any combination of text that for the filter parameter that doesn't end with vitality goes saying "No Images Found". I'm using @JVital2013 's additions to dataHandler and functions shown above.

My path directory is: /home/pi/sats/leosats

An example of what isn't showing any images:

[allnoaa_mcir]
path= "/home/pi/sats/leosats"
title = "NOAA 15"
filter = "[\\\/]avhrr"
mode = "satdump_leo"

Any thoughts as to why filtering isn't working? Thanks!

@ImDroided
Copy link

ImDroided commented Oct 12, 2024 via email

@JC167shearer
Copy link

I don't know if this helps the troubleshooting or not, but couple more things I've noticed. If I set fast = true, zero images are returned. If fast = false, it returns to displaying all images.

Additionally, it's really not displaying all images. Rather, it shows me the first image (msu_mr_rgb_MSA_corrected_map.png for example, from my most recent Meteor pass), eleven times. There are eleven images in directory /home/pi/sats/leosats/2024-10-13_10-00_meteor_m2-x_lrpt_137.9 MHz/MSU-MR, but the page is showing me the first image (MSA_corrected_map) eleven times. That is, when I click the image and it gets larger to view, and you have the right and left arrows... I have to click the left arrow eleven times before the page displays the previous satellite pass.

I don't really understand how the image arrays are built, but it looks like only the first image is being put into the array X number of times... like something with the array builder isn't incrementing. I'm grasping at straws; I don't know php at all...

Hopefully this helps?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants