Multiple paths required

[kesterlester]

When building from source on my mac, I found I needed to modify more build.xml files than I anticipated. I've listed some of them below. With enough such changes I can indeed build Jorgan and it makes noises and I can load some dispositions from elsewhere. So in that sense there is no "issue" at all.

However it also feels to me like there should be a few more options in build.properties which can feed down in a more natural way to the sub-packages so that other people in future building on mac don't have to solve the same problems I had to investigate.

I am hoping that perhaps by posing an issue here it might be possible to work toward such a resolution.

E.g. having set
mac.include = /Library/Java/JavaVirtualMachines/jdk-11.0.12.jdk/Contents/Home/include
in build.properties (which I did expect to change) I still had to add extra lines to jorgan-fluidsynth/build.xml as shown in the diff below.

--- a/jorgan-fluidsynth/build.xml
+++ b/jorgan-fluidsynth/build.xml
@@ -141,6 +141,8 @@
       <arg value="-dynamiclib" />
       <arg value="-std=c99" />
       <arg value="-I${mac.include}" />
+      <arg value="-I${mac.include}/darwin" /> <!-- LESTER -->
+      <arg value="-I/opt/local/include" /> <!-- LESTER : for fluidsynth -->
       <arg value="-I../jorgan-jni/src/main/native" />
       <arg value="-I./lib/include" />
       <arg value="-I./target/native" />

As you can see there are two extra lines above.

The FIRST additional line was because the header files installed in $JAVA_HOME/include by the https://download.oracle.com/otn-pub/java/jdk/11.0.12+8/f411702ca7704a54a79ead0c2e0942a3/jdk-11.0.12_osx-x64_bin.dmg installer oracle had me use contain files which are used by organ with names like:

${mac.include}/jni.h

which themselves contain lines like

#include "jni_md.h"

despite "jni_md.h" actually being one directory deeper in the darwin sub-folder. Presumably this is because jni_md.h contains system dependent code.

THIS FIRST ISSUE would be solved if build.properties contained some facility for me to include multiple include paths, not just a single one, for mac.include. Then I could include both my $JAVA_HOME/include and $JAVA_HOME/include/darwin. I did initially hope I might be able to do so by using some magic separator (like colons), e,g. setting:

mac.include=$JAVA_HOME/include:$JAVA_HOME/include/darwin

however the above (even if it were compatible with the java compile line, which I am not 100% sure it is) would still trigger a file-not-found failure in places in jorgan-fluidsynth/build.xml where tests like this are done:

    <fail message="mac.include not found - please check your build.properties">
      <condition>
        <not>
          <available file="${mac.include}" />
        </not>
      </condition>
    </fail>

I am neither a java nor an ant expert, so rather than attempt to fix the test above I just hard-coded extra include paths into the build line as shown in the original diff. However, it feels to me that someone who is a java/ant expert would be able to propose a better fix than the one I did from the get-go.

The second line you can see I added to jorgan-fluidsynth/build.xml was so that the jOrgan build could find my system's fluidsynth headers. Again, I just added the line so that the thing would compile, but I presume that someone more au-fait with ant would propose a more sensible high-level place to put such paths in a way that would be more maintainable for others in future?

In case anyone wants to see the sum total of all the changes I made, they can be seen in the "LESTER_M1_CONFIG" branch of my Jorgan fork here:
https://github.com/kesterlester/jorgan/tree/LESTER_M1_CONFIG

[svenmeier]

Does this still hold for Java 16?

[kesterlester]

Perhaps it was not a good idea for me to have mentioned two issues in this "issue". Both are related to include paths, but the fluidsynth-header issue is technically unrelated to the java-native include issue, so perhaps I should have kept them separate. Treating them separately below:

===========================================
On the java-native include issue ...

(This was my need to add a
<arg value="-I${mac.include}/darwin" /> <!-- LESTER -->
line after
<arg value="-I${mac.include}" />
in jorgan-fluidsynth/build.xml )

Yes this tweak appears to be needed in ALL the versions of JDK I installed on my mac, whether V11 or V16. If I omit my extra line above the jorgan-fluidsynth compilation fails with:

[native2ascii] Converting 12 files from /Users/lester/JORGAN/sources/jorgan-fluidsynth/src/main/java to /Users/lester/JORGAN/sources/jorgan-fluidsynth/target/classes
     [copy] Copying 13 files to /Users/lester/JORGAN/sources/jorgan-fluidsynth/target/classes
     [copy] Copying 1 file to /Users/lester/JORGAN/sources/jorgan-fluidsynth/target/classes

native-common:

native-unix:

native-windows:

native-mac:
     [exec] In file included from ../jorgan-jni/src/main/native/exception.c:1:
     [exec] /Library/Java/JavaVirtualMachines/jdk-16.0.2.jdk/Contents/Home/include/jni.h:45:10: fatal error: 'jni_md.h' file not found
     [exec] #include "jni_md.h"
     [exec]          ^~~~~~~~~~
     [exec] 1 error generated.
     [exec] In file included from ../jorgan-jni/src/main/native/logging.c:1:
     [exec] /Library/Java/JavaVirtualMachines/jdk-16.0.2.jdk/Contents/Home/include/jni.h:45:10: fatal error: 'jni_md.h' file not found
     [exec] #include "jni_md.h"
     [exec]          ^~~~~~~~~~
     [exec] 1 error generated.
     [exec] In file included from ./src/main/native/fluidsynthJNI.c:1:
     [exec] /Library/Java/JavaVirtualMachines/jdk-16.0.2.jdk/Contents/Home/include/jni.h:45:10: fatal error: 'jni_md.h' file not found
     [exec] #include "jni_md.h"
     [exec]          ^~~~~~~~~~
     [exec] 1 error generated.

BUILD FAILED
/Users/lester/JORGAN/sources/build.xml:12: The following error occurred while executing this line:
/Users/lester/JORGAN/sources/jorgan-fluidsynth/build.xml:140: exec returned: 1

and the reason is that all three versions of Java I installed (a V11 via dmg download from Oracle, a V11 openssl via "macports install", and a V16 via dmg download from Oracle) share a common header directory under $JAVA_HOME/include structure like this:

$JAVA_HOME/include/classfile_constants.h
$JAVA_HOME/include/jawt.h
$JAVA_HOME/include/jdwpTransport.h
$JAVA_HOME/include/jni.h
$JAVA_HOME/include/jvmti.h
$JAVA_HOME/include/jvmticmlr.h
$JAVA_HOME/include/darwin/jawt_md.h
$JAVA_HOME/include/darwin/jni_md.h

with (for example) $JAVA_HOME/include/jni.h containing the following include for $JAVA_HOME/include/darwin/jni_md.h with out a mention of the darwin subdirectory:

/* jni_md.h contains the machine-dependent typedefs for jbyte, jint
   and jlong */

#include "jni_md.h"

I am guessing that there is a very standard convention or work pattern that the above directory stricture is supposed to be used with, to permit java code that's independent of machine-dependencies. I just don't know what the intended convention is ...

Of course, BEST OF ALL would be if it turned out that no dependency on jni.h were actually needed. I have not tried to work out what is using it and whether it's actually still used. I could look.

===========================================
On the fluidsynth include issue ...

(This was my need to add a
<arg value="-I/opt/local/include" /> <!-- LESTER : for fluidsynth -->
line after
<arg value="-I${mac.include}" />
in jorgan-fluidsynth/build.xml )

The above need is entirely separate from java version. It's just needed to find the fluidsynth headers, which (presumably) could be in many different places for many different people, even on (say) "just" mac. E.g. I installed fluidsynth with "macports" which put its headers in /opt/local/include/fluidsynth.... but some other person could have used "home-brew" which would have put the headers elsewhere, or someone could instead have choosen to build fluidsynth from source by hand and put the headers almost anywhere. Presumably there are similar issues in linux and windows.

Without my temporary fix above the jorgan-fluidsynth build fails with:

native-common:

native-unix:

native-windows:

native-mac:
     [exec] In file included from ./src/main/native/fluidsynthJNI.c:7:
     [exec] ./lib/include/fluidsynth.h:111:10: fatal error: 'fluidsynth/ladspa.h' file not found
     [exec] #include "fluidsynth/ladspa.h"
     [exec]          ^~~~~~~~~~~~~~~~~~~~~
     [exec] 1 error generated.

BUILD FAILED
/Users/lester/JORGAN/sources/build.xml:12: The following error occurred while executing this line:
/Users/lester/JORGAN/sources/jorgan-fluidsynth/build.xml:140: exec returned: 1

Total time: 25 seconds

---

My 'loose' feelings are that (1) maybe the best solution is to change the way paths are specified in boot.properties so that a LIST of include paths, rather than a SINGLE include path can be passed on, and (2) I imagine that the issue is relevant for all operating systems not just mac.

So, maybe boot.properties should be changed from accepting only single paths at present:

win.cc = C:/MinGW32/bin/gcc.exe
win.include = C:/Programme/Java/jdk1.7.0_09/include

unix.cc = gcc
unix.include = /usr/lib/jvm/default-java/include

mac.cc = gcc
mac.include = /Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Headers

to instead allowing lists:

win.cc = C:/MinGW32/bin/gcc.exe
win.include = C:/Programme/Java/jdk1.7.0_09/include , SOMEWHERE_ELSE , ANOTHER_IF_NEEDED , ....

unix.cc = gcc
unix.include = /usr/lib/jvm/default-java/include , SOMEWHERE_ELSE , ANOTHER_IF_NEEDED , ....

mac.cc = gcc
mac.include = /Developer/SDKs/MacOSX10.6.sdk/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Headers , SOMEWHERE_ELSE , ANOTHER_IF_NEEDED , ....

and then the relevant build.xml files would expand each list item into a separate inclusion. However, this idea may be torpedoed if (a) no list structures permitted in boot.properties , or (b) even if there are, there is no mechanism to expand a list into multiple lines in a build.xml:

<arg value="-I${mac.include ELEMENT 1}" />
<arg value="-I${mac.include ELEMENT 2 }" />
<arg value="-I${mac.include ELEMENT 3 }" />
....

In either such case a fallback could be to leverage any mechanisms in the java compiler's input argument processor that accept multiple (e.g. colon separated) include paths on its own command line. The separator may vary from OS to OS, but that's OK. However, if one used such a mechanism (i.e. permitting

mac.include = SOMEWHERE:SOMEWHERE_ELSE:ANOTHER_IF_NEEDED

in build.properties, then this would break some "does this file exist" tests that are done in, e.g., jorgan-fluidsynth/build.xml :

    <fail message="mac.include not found - please check your build.properties">
      <condition>
        <not>
          <available file="${mac.include}" />
        </not>
      </condition>
    </fail>

which would need to be changed accordingly.

[kesterlester]

Hmm. This is interesting: jni.h is included in a few places:

Christophers-M1:sources lester$ find . -name "*.h" | xargs grep jni.h
./jorgan-fluidsynth/target/native/jorgan_fluidsynth_Fluidsynth.h:#include <jni.h>
./jorgan-bundle/target/jOrgan-4.0-beta2/jOrgan.app/Contents/Resources/Java/lib/jorgan_fluidsynth_Fluidsynth.h:#include <jni.h>
./jorgan-creative/target/native/jorgan_creative_SoundFontManager.h:#include <jni.h>
./jorgan-jni/src/main/native/logging.h:#include <jni.h>
./jorgan-jni/src/main/native/exception.h:#include <jni.h>

BUT I only had to add a workaround the inclusion in jorgan-fluidsynth. That presumably means that the other cases already have good (non-hack) workarounds that can be moved over to fluidsynth?? I will investigate.

[kesterlester]

Hmm:

• jorgan-jni doesn't build anything
• jorgan-creative has native-linux and native-windows targets but no native-mac targets
• jorgan-bundle is doing something else.

So all of the above didn't really include jni.h in the same way that fluidsynth does.

HOWEVER

I note that whoever wrote the following lines of jorgan-creative/build.xml

  <target name="native-windows" depends="native-common" if="isWindows">
  ... snip ...
    <exec failonerror="true" executable="${win.cc}">
      <arg value="-std=c99" />
      <arg value="-I${win.include}" />
      <arg value="-I${win.include}/win32" />
      <arg value="-I../jorgan-jni/src/main/native" />
      <arg value="-I./lib/win/include" />
      <arg value="-I./target/native" />
      <arg value="-Wall" />
      <arg value="-Wl,--kill-at" />
      <arg value="-shared" />
      <arg line="../jorgan-jni/src/main/native/exception.c" />
      <arg value="./src/main/native/win/creativeJNI.c" />
      <arg line="-o ./target/native/creativeJNI.dll" />
    </exec>
... snip...
   </target>

felt entirely happy having the lines:

    <arg value="-I${win.include}" />
    <arg value="-I${win.include}/win32" />

inside a win-dependent target. Later on in a windows dependent target the same file is happy to have a pair of lines:

<arg value="-I${unix.include}" />
<arg value="-I${unix.include}/linux" />

I am therefore beginning to infer that my originally reported workaround is (adding the darwin director) is actually "in keeping" with the expected practice.

I will therefore make a pull-request for the adding of the darwin line.

It still remains to figure out what to do with the fluidsynth include issue.

[kesterlester]

OK - I made a pull-request for the addition of the darwin line. Find it here:
#6

However this still leaves open the question of the most appropriate way of making sure that the build can find "fluidsynth/ladspa.h"