question about fourier transform

[dragonyanglong]

Dear @rayosborn , while using nexpy for 3D-dPDF analysis, I have a question about the fourier transform applied in the nxreduce step.

https://github.com/rayosborn/nxrefine/blob/6de6d28a3f63bd95ae7d67c58f455cae0e233d04/src/nxrefine/nxreduce.py#L2230

        fft = np.real(scipy.fft.fftshift(
            scipy.fft.fftn(scipy.fft.fftshift(symm_data[:-1, :-1, :-1]),
                           workers=self.process_count)))

Question 1: why the last slice of data is cropped? symm_data[:-1, :-1, :-1] instead of symm_data?

Question 2: why using fftshift twice? I thought we usually use scipy.fft.ifftshift(scipy.fft.fftn(scipy.fft.fftshift))?
I made some quick tests, and the following sequences give different results, which one would be correct?

• scipy.fft.fftshift(scipy.fft.fftn(scipy.fft.fftshift(data)))
• scipy.fft.ifftshift(scipy.fft.fftn(scipy.fft.ifftshift(data)))
• scipy.fft.ifftshift(scipy.fft.fftn(scipy.fft.fftshift(data)))
• scipy.fft.fftshift(scipy.fft.fftn(scipy.fft.ifftshift(data)))

Thanks!

[krogstad]

Dear Long, You've identified a copy of idiosyncrasies in our implementation. For 1), the thought was that FFTs work best when there is an even number of points, and with the origin being at the center of the central bin, there will always be an odd number of points for each axis. As I recall, I initially took this advice from other 3DPDF users, but this didn't actually change much when I tested it (the use of a windowing function further reduces the relevance of this choice). As for FFT vs inverse FFT: for a real and symmetric dataset (as any scattering data should be), these operations should be equivalent up to a scaling factor, and if they are not, this indicates a problem with the input data. (If you are trying to connect intensities in the deltaPDF to absolute units, this would be an important distinction.) Best, Matt

…

________________________________ From: longyang ***@***.***> Sent: Tuesday, May 21, 2024 4:12 AM To: nexpy/nxrefine ***@***.***> Cc: Subscribed ***@***.***> Subject: [nexpy/nxrefine] question about fourier transform (Issue #103) Dear @rayosborn , while using nexpy for 3D-dPDF analysis, I have a question about the fourier transform applied in the nxreduce step. https: //github. com/rayosborn/nxrefine/blob/6de6d28a3f63bd95ae7d67c58f455cae0e233d04/src/nxrefine/nxreduce. py#L2230 ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Dear @rayosborn<https://urldefense.us/v3/__https://github.com/rayosborn__;!!G_uCfscf7eWS!bm29X_ufGs0sT8XzWcTYxwwM5HU88pm-S2ItMKX7vo0gSkZVK7nVxtzj0xvuh25hzwpgVZ9OFnyL_u_wDSpukHmW$> , while using nexpy for 3D-dPDF analysis, I have a question about the fourier transform applied in the nxreduce step. https://github.com/rayosborn/nxrefine/blob/6de6d28a3f63bd95ae7d67c58f455cae0e233d04/src/nxrefine/nxreduce.py#L2230<https://urldefense.us/v3/__https://github.com/rayosborn/nxrefine/blob/6de6d28a3f63bd95ae7d67c58f455cae0e233d04/src/nxrefine/nxreduce.py*L2230__;Iw!!G_uCfscf7eWS!bm29X_ufGs0sT8XzWcTYxwwM5HU88pm-S2ItMKX7vo0gSkZVK7nVxtzj0xvuh25hzwpgVZ9OFnyL_u_wDdj9O3Fl$> fft = np.real(scipy.fft.fftshift( scipy.fft.fftn(scipy.fft.fftshift(symm_data[:-1, :-1, :-1]), workers=self.process_count))) Question 1: why the last slice of data is cropped? symm_data[:-1, :-1, :-1] instead of symm_data? Question 2: why using fftshift twice? I thought we usually use scipy.fft.ifftshift(scipy.fft.fftn(scipy.fft.fftshift))? I made some quick tests, and the following sequences give different results, which one would be correct? * scipy.fft.fftshift(scipy.fft.fftn(scipy.fft.fftshift(data))) * scipy.fft.ifftshift(scipy.fft.fftn(scipy.fft.ifftshift(data))) * scipy.fft.ifftshift(scipy.fft.fftn(scipy.fft.fftshift(data))) * scipy.fft.fftshift(scipy.fft.fftn(scipy.fft.ifftshift(data))) Thanks! — Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https://github.com/nexpy/nxrefine/issues/103__;!!G_uCfscf7eWS!bm29X_ufGs0sT8XzWcTYxwwM5HU88pm-S2ItMKX7vo0gSkZVK7nVxtzj0xvuh25hzwpgVZ9OFnyL_u_wDb3kBz-v$>, or unsubscribe<https://urldefense.us/v3/__https://github.com/notifications/unsubscribe-auth/ABFSH7FLLRFUBO2QPI2KDDDZDMFYDAVCNFSM6AAAAABIBHBWE6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGMYDONZYGY4DSNQ__;!!G_uCfscf7eWS!bm29X_ufGs0sT8XzWcTYxwwM5HU88pm-S2ItMKX7vo0gSkZVK7nVxtzj0xvuh25hzwpgVZ9OFnyL_u_wDT-ADNZb$>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[dragonyanglong]

Dear Matt,

Thanks so much for your detailed response. I did some further testing. I use the real experimental data collected at beamline, and tested even/odd and fftshift/iffshit. The result is attached. We found that:

When the input data shape is even number like (200, 200, 200), the results of fftshift(fftn(fftshift(data)))) and fftshift(fftn(ifftshift(data)))) are the same.
When the input data shape are odd numbers like (201, 201, 201), the results are different.

I think we usually do shift and inverse shift together when implementing Fourier transform, but here in nxreduce, we do shift twice, no inverse shift. So that's why I opened this issue.

Thanks Matt.

Best,
Long

[krogstad]

Dear Long, That’s important, thanks for bringing this up! I’ll look into this a bit more . . . My guess is that with an odd number of inputs, the zero/max/min frequency bins don’t end up in the expected spaces, causing an issue. Numpy documentation says that fftshift and ifftshift are identical for an even number of bins but not for an odd number of bins, and since we were blithely forcing an even number of bins, we didn’t notice the issue. Best, Matt From: longyang ***@***.***> Sent: Wednesday, May 22, 2024 9:31 PM To: nexpy/nxrefine ***@***.***> Cc: Krogstad, Matthew ***@***.***>; Comment ***@***.***> Subject: Re: [nexpy/nxrefine] question about fourier transform (Issue #103) Dear Matt, Thanks so much for your detailed response. I did some further testing. I use the real experimental data collected at beamline, and tested even/odd and fftshift/iffshit. The result is attached. We found that: When the input data shape ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. ZjQcmQRYFpfptBannerEnd Dear Matt, Thanks so much for your detailed response. I did some further testing. I use the real experimental data collected at beamline, and tested even/odd and fftshift/iffshit. The result is attached. We found that: When the input data shape is even number like (200, 200, 200), the results of fftshift(fftn(fftshift(data)))) and fftshift(fftn(ifftshift(data)))) are the same. When the input data shape are odd numbers like (201, 201, 201), the results are different. I think we usually do shift and inverse shift together when implementing Fourier transform, but here in nxreduce, we do shift twice, no inverse shift. So that's why I opened this issue. Thanks Matt. Best, Long image.png (view on web)<https://urldefense.us/v3/__https:/github.com/nexpy/nxrefine/assets/16789768/dbfb76d1-4b50-4276-8bca-85ff7931a120__;!!G_uCfscf7eWS!bp2Xo87Yz-tSIgsAJoFIRZgAx9MOdxqLr8IjqcJAE6sotG4PDcf-_2sA45oq9xb1uL-fphLSjRLZosqxZPTyMGWD$> — Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https:/github.com/nexpy/nxrefine/issues/103*issuecomment-2126077356__;Iw!!G_uCfscf7eWS!bp2Xo87Yz-tSIgsAJoFIRZgAx9MOdxqLr8IjqcJAE6sotG4PDcf-_2sA45oq9xb1uL-fphLSjRLZosqxZLdPuGAZ$>, or unsubscribe<https://urldefense.us/v3/__https:/github.com/notifications/unsubscribe-auth/ABFSH7E7RSGZCHTZHVICJ7DZDVIFHAVCNFSM6AAAAABIBHBWE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRWGA3TOMZVGY__;!!G_uCfscf7eWS!bp2Xo87Yz-tSIgsAJoFIRZgAx9MOdxqLr8IjqcJAE6sotG4PDcf-_2sA45oq9xb1uL-fphLSjRLZosqxZA_V2a1z$>. You are receiving this because you commented.Message ID: ***@***.******@***.***>>