Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: PERF: Performance acceleration via GPU-accelerated FFT's #1331

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

WIP: PERF: Performance acceleration via GPU-accelerated FFT's #1331

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0a48f1b
ENH: Bump ITK to 2022-04-05 master
thewtex Apr 6, 2022
e02fee4
COMP: Undefined ITK_COORDINATE_ORIENTATION_INVALID
thewtex Apr 6, 2022
325f5c5
WIP: PERF: Add GPU-accelerated Gaussian smoothing support
thewtex Apr 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 58 additions & 1 deletion ANTS.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,64 @@ set(CPACK_RESOURCE_FILE_README "${CMAKE_CURRENT_SOURCE_DIR}/README.txt")
message(STATUS "Building ${PROJECT_NAME} version \"${${PROJECT_NAME}_VERSION}\"")

# Set up ITK
find_package(ITK ${ITK_VERSION_ID} REQUIRED)
find_package(ITK ${ITK_VERSION_ID} REQUIRED
Copy link
Member

@cookpa cookpa Apr 6, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block breaks the default SuperBuild for me. It's asking for ITK components that aren't built by default:

VkFFTBackend
ITKFEM
ITKVTK
ITKVtkGlue

I don't know about ITKFEM, but the others should be required only if the relevant build options are turned on.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ITKFEM is because Temporary/ has includes which use it.

VkFFTBackend is the code which supports this new feature.

ITKVTK and ITKVTK need to be wrapped so they only are required if the USE_VTK is on.

I'm working on this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also Examples/ConformalMapping.cxx and Examples/ANTSConformalMapping.cxx but it appears these are not built, since ITKFEM module hasn't been enabled and the build still works.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was working on this as well, but I could not build with vkFFT support. It stops with this error

make                                                                                                                                                                   
[ 47%] Built target ITKv5
[ 52%] Performing forcebuild step for 'ANTS'
[ 58%] Performing build step for 'ANTS'
[  1%] Building CXX object Examples/CMakeFiles/antsUtilities.dir/antsRegistrationTemplateHeader.cxx.o
In file included from /Users/pcook/tmp/NOT_BACKED_UP/antsGPU/ANTs/Examples/antsRegistrationTemplateHeader.cxx:1:
In file included from /Users/pcook/tmp/NOT_BACKED_UP/antsGPU/ANTs/Examples/antsRegistrationTemplateHeader.h:14:
In file included from /Users/pcook/tmp/NOT_BACKED_UP/antsGPU/ANTs/Examples/itkantsRegistrationHelper.h:1472:
In file included from /Users/pcook/tmp/NOT_BACKED_UP/antsGPU/ANTs/Examples/itkantsRegistrationHelper.hxx:7:
In file included from /Users/pcook/tmp/NOT_BACKED_UP/antsGPU/build/staging/include/ITK-5.3/itkVkHalfHermitianToRealInverseFFTImageFilter.h:24:
/Users/pcook/tmp/NOT_BACKED_UP/antsGPU/build/staging/include/ITK-5.3/itkVkCommon.h:24:10: fatal error: 'vkFFT.h' file not found
#include "vkFFT.h"
         ^~~~~~~~~
1 error generated.
make[5]: *** [Examples/CMakeFiles/antsUtilities.dir/antsRegistrationTemplateHeader.cxx.o] Error 1
make[4]: *** [Examples/CMakeFiles/antsUtilities.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [ANTS-prefix/src/ANTS-stamp/ANTS-build] Error 2
make[1]: *** [CMakeFiles/ANTS.dir/all] Error 2
make: *** [all] Error 2

Have you been able to compile with the vtFFT module on?

COMPONENTS
VkFFTBackend
ITKAnisotropicSmoothing
ITKAntiAlias
ITKBiasCorrection
ITKBinaryMathematicalMorphology
ITKClassifiers
ITKColormap
ITKCommon
ITKConnectedComponents
ITKConvolution
ITKDisplacementField
ITKDistanceMap
ITKFEM
ITKFastMarching
ITKFiniteDifference
ITKIOCSV
ITKIOGDCM
ITKIOImageBase
ITKIONRRD
ITKIOTransformBase
ITKImageAdaptors
ITKImageCompare
ITKImageFeature
ITKImageFilterBase
ITKImageFunction
ITKImageGradient
ITKImageGrid
ITKImageIntensity
ITKImageLabel
ITKImageNoise
ITKImageSources
ITKImageStatistics
ITKLabelMap
ITKMarkovRandomFieldsClassifiers
ITKMathematicalMorphology
ITKMesh
ITKMetricsv4
ITKOptimizers
ITKOptimizersv4
ITKPDEDeformableRegistration
ITKPolynomials
ITKRegistrationCommon
ITKRegistrationMethodsv4
ITKReview
ITKSmoothing
ITKSpatialObjects
ITKStatistics
ITKTestKernel
ITKThresholding
ITKTransform
ITKTransformFactory
ITKVTK
ITKVtkGlue
ITKImageIO
ITKTransformIO
)
include(${ITK_USE_FILE})

# Set up which ANTs apps to build
Expand Down
2 changes: 1 addition & 1 deletion Examples/PrintHeader.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ PrintHeader(int argc, char * argv[])
// Get the metadata as a generic object
string key = itMeta->first, v_string;
itk::SpatialOrientation::ValidCoordinateOrientationFlags v_oflags =
itk::SpatialOrientation::ITK_COORDINATE_ORIENTATION_INVALID;
itk::SpatialOrientationEnums::ValidCoordinateOrientations::ITK_COORDINATE_ORIENTATION_INVALID;

if (itk::ExposeMetaData<string>(mdd, key, v_string))
{
Expand Down
1 change: 1 addition & 0 deletions Examples/itkantsRegistrationHelper.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
#include "itkEuler2DTransform.h"
#include "itkEuler3DTransform.h"
#include "itkExpectationBasedPointSetToPointSetMetricv4.h"
#include "itkFFTDiscreteGaussianImageFilterFactory.h"
#include "itkGaussianExponentialDiffeomorphicTransform.h"
#include "itkGaussianExponentialDiffeomorphicTransformParametersAdaptor.h"
#include "itkGaussianSmoothingOnUpdateDisplacementFieldTransform.h"
Expand Down
14 changes: 14 additions & 0 deletions Examples/itkantsRegistrationHelper.hxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@
#include <vnl/vnl_matrix.h>
#include <vnl/vnl_copy.h>

#include "itkVkHalfHermitianToRealInverseFFTImageFilter.h"
#include "itkVkRealToHalfHermitianForwardFFTImageFilter.h"
#include "itkFFTImageFilterFactory.h"

namespace ants
{

Expand Down Expand Up @@ -65,6 +69,16 @@ RegistrationHelper<TComputeType, VImageDimension>::RegistrationHelper()
typedef itk::LinearInterpolateImageFunction<ImageType, RealType> LinearInterpolatorType;
typename LinearInterpolatorType::Pointer linearInterpolator = LinearInterpolatorType::New();
this->m_Interpolator = linearInterpolator;

using ForwardFactoryType = itk::FFTImageFilterFactory<itk::VkRealToHalfHermitianForwardFFTImageFilter>;
typename ForwardFactoryType::Pointer forwardFactory{ ForwardFactoryType::New() };
itk::ObjectFactoryBase::RegisterFactory(forwardFactory, itk::ObjectFactoryEnums::InsertionPosition::INSERT_AT_FRONT);

using InverseFactoryType = itk::FFTImageFilterFactory<itk::VkHalfHermitianToRealInverseFFTImageFilter>;
typename InverseFactoryType::Pointer inverseFactory{ InverseFactoryType::New() };
itk::ObjectFactoryBase::RegisterFactory(inverseFactory, itk::ObjectFactoryEnums::InsertionPosition::INSERT_AT_FRONT);

itk::FFTDiscreteGaussianImageFilterFactory::RegisterOneFactory();
}

template <typename TComputeType, unsigned VImageDimension>
Expand Down
2 changes: 2 additions & 0 deletions SuperBuild.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ CMAKE_DEPENDENT_OPTION(

option(ITK_BUILD_MINC_SUPPORT "Build support for MINC2" OFF)

option(ITK_BUILD_VKFFT_SUPPORT "Build support for VkFFT GPU accelerated FFTs" OFF)

set(EXTERNAL_PROJECT_BUILD_TYPE "Release" CACHE STRING "Default build type for support libraries")


Expand Down
11 changes: 9 additions & 2 deletions SuperBuild/External_ITKv5.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@ if(NOT DEFINED ${extProjName}_DIR AND NOT ${USE_SYSTEM_${extProjName}})
-DModule_ITKMINC:BOOL=ON
)
endif()
if(${extProjName}_BUILD_VKFFT_SUPPORT)
set(${proj}_MINC_ARGS
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be "${proj}_VKFFT_ARGS"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't get to testing this build and fixing it up yet.

-DModule_VkFFTBackend:BOOL=ON
)
endif()

set(${proj}_WRAP_ARGS)
#if(foo)
Expand Down Expand Up @@ -136,6 +141,7 @@ if(NOT DEFINED ${extProjName}_DIR AND NOT ${USE_SYSTEM_${extProjName}})
${${proj}_FFTWF_ARGS}
${${proj}_FFTWD_ARGS}
${${proj}_MINC_ARGS}
${${proj}_VKFFT_ARGS}
)

if( USE_VTK STREQUAL "ON" )
Expand All @@ -151,9 +157,10 @@ if(NOT DEFINED ${extProjName}_DIR AND NOT ${USE_SYSTEM_${extProjName}})


### --- End Project specific additions
set(${proj}_REPOSITORY ${git_protocol}://github.com/InsightSoftwareConsortium/ITK.git)
# set(${proj}_REPOSITORY ${git_protocol}://github.com/InsightSoftwareConsortium/ITK.git)
set(${proj}_REPOSITORY ${git_protocol}://github.com/thewtex/ITK.git)
# set(${proj}_REPOSITORY ${git_protocol}://github.com/stnava/ITK.git)
set(${proj}_GIT_TAG v5.3rc03) # Official ITK release version 5.3.0 rc 03 # Many modern code updates for performance
set(${proj}_GIT_TAG d5abe809e26043e888a887d4dafc39fabb0fb2b3) # Official ITK release version 5.3.0 rc 03 # Many modern code updates for performance
set(ITK_VERSION_ID ITK-5.3) ### NOTE: When updating GIT_TAG, also update ITK_VERSION_ID

ExternalProject_Add(${proj}
Expand Down