将 native library 打包进 jar 里
有一个 Java 写的程序，需要在 Spark 上跑，所以运行之前没法设置环境变量，也没办法设置 Java 运行参数（不知道是否真的不能设置，但甲方这么说，无奈啊），需要在运行的时候把 native library 加载起来。之前没有怎么写过 Java ，琢磨了一下反正也写出来了。
Tensorflow Java 1.14.0 的 bug
Tensorflow 在运行的时候， Linux 报：
Exception in thread 'main' java.lang.UnsatisfiedLinkError: /tmp/tensorflow_native_libraries-1680337452521-0/libtensorflow_jni.so: libtensorflow_framework.so.1: 无法打开共享文件：没有那个文件或目录
at ...
at java.lang.System.load(System.java:1100)
at org.tensorflow.NativeLibrary.load(NativeLibrary.java:101)
at ...
看这个 PR tensorflow/tensorflow#32829 ， tensorflow 解压了 libtensorflow_framework.so ，但 libtensorflow_jni.so 链接到的是 libtensorflow_framework.so.1 这个文件。
在这个版本的话，要想修复这个问题有很多种办法，比如用 maven-plugin-shade 在打包的时候把这个类替换掉。但是甲方还是不同意～另一个简单的方法就是抢先在 tensorflow 初始化前先把 libtensorflow_framework.so 加载进来，同 tf 加载的套路一样，从资源文件 jar 里把 so 解压出来，然后调用 System.load 加载就 OK 了。
JSCIP 的编译和打包
之所以之前那么加载是 OK 的，是因为如果一个动态链接库已经被 Java 加载了，那么依赖它的动态链接库就可以直接用内存中的这个库，而不会再重新从文件系统中再加载。这就意味着不能直接用 conda-forge 打包的 SCIP ，或者是直接用 conda-forge 的工具链去编译。（倒不是说 conda-forge 的工具链不能编译，而是 conda-forge 里包的依赖关系做得太烂，很多 native 的包依赖高版本 glibc ，而没有把 sysroot 写进依赖里。）
目标机器是 CentOS 7 ，为了简单本地搭了一下环境：
- GCC 9 从 centos scl 里来，虽然最后 C++ 遇到了 C++11 ABI 不兼容的问题，但貌似问题不大（为啥啊？）
- CMake 3 从 EPEL 里来
- Boost 从 Boost 网站上来
- TBB 从 GitHub 上来
- 其他一些依赖也从 EPEL 里来
cmake .. -DTBB_DIR="$(pwd)/../../tbb2019_20190320oss" -DBOOST_INCLUDEDIR=/home/azuk/sciptest/scip2/boost_1_81_0 -DBOOST_LIBRARYDIR=/home/azuk/sciptest/scip2/boost_1_81_0/libs
编译 papilo 报错了，但是不想研究为啥，因为 libscip 编译出来了。但是 SCIP 不应该依赖 SoPlex 吗，我不理解啊。。
之后就是 ldd 把依赖找出来，打进一个 jar 包里，等需要用的时候解压出来就行了。
参考了 tensorflow 的 NativeLibrary.java 瞎编的：
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
public class NativeLibrary {
private static boolean loaded = false;
private static File tempPath = null;
public static void load() {
if (loaded) return;
log(String.format("classpath: %s", System.getProperty ("java.class.path")));
switch (os()) {
case "windows":
loadLibrary("tbb.dll");
loadLibrary("libscip.dll");
loadLibrary("jscip.dll");
break;
case "linux":
loadLibrary("libtinfo.so.5");
loadLibrary("libz.so.1");
loadLibrary("libpord-5.3.so");
loadLibrary("libquadmath.so.0");
loadLibrary("libgfortran.so.3");
loadLibrary("libgmp.so.10");
loadLibrary("libgmpxx.so.4");
loadLibrary("libblas.so.3");
loadLibrary("libbz2.so.1");
loadLibrary("libdl.so.2");
loadLibrary("libreadline.so.6");
loadLibrary("libtbb.so.2");
loadLibrary("libgomp.so.1");
loadLibrary("libscotcherr.so.0");
loadLibrary("libscotch.so.0");
loadLibrary("libmetis.so.0");
loadLibrary("libmpiseq-5.3.so");
loadLibrary("libesmumps.so.0");
loadLibrary("libscotchmetis.so.0");
loadLibrary("libopenblas.so.0");
loadLibrary("libmumps_common-5.3.so");
loadLibrary("libdmumps-5.3.so");
loadLibrary("libipopt.so.3");
loadLibrary("libscip.so.8.0");
loadLibrary("libjscip.so");
break;
}
loaded = true;
}
private static void log(String msg) {
System.err.println("NativeLibrary: " + msg);
}
private static void loadLibrary(String libName) {
final String jniLibName = libName;
log(String.format("Load library %s", jniLibName));
final String resourceName = makeResourceName(jniLibName);
InputStream jniResource = NativeLibrary.class.getResourceAsStream(resourceName);
if (jniResource == null) {
jniResource = NativeLibrary.class.getClass().getResourceAsStream(resourceName);
}
if (jniResource == null) {
throw new UnsatisfiedLinkError("Native library not found");
}
String libPath;
try {
if (tempPath == null) {
tempPath = createTemporaryDirectory();
tempPath.deleteOnExit();
}
final String tempDirectory = tempPath.getCanonicalPath();
libPath = extractResource(jniResource, jniLibName, tempDirectory);
} catch (IOException e) {
throw new UnsatisfiedLinkError("Unable to extract native library.");
}
System.load(libPath);
}
private static String os() {
final String p = System.getProperty("os.name").toLowerCase();
if (p.contains("linux")) {
return "linux";
} else if (p.contains("os x") || p.contains("darwin")) {
return "darwin";
} else if (p.contains("windows")) {
return "windows";
} else {
return p.replaceAll("\\s", "");
}
}
private static String architecture() {
final String arch = System.getProperty("os.arch").toLowerCase();
return (arch.equals("amd64")) ? "x86_64" : arch;
}
private static String makeResourceName(String baseName) {
return String.format("/%s-%s/", os(), architecture()) + baseName;
}
private static String extractResource(
InputStream resource, String resourceName, String extractToDirectory) throws IOException {
final File dst = new File(extractToDirectory, resourceName);
dst.deleteOnExit();
final String dstPath = dst.toString();
final long nbytes = copy(resource, dst);
return dstPath;
}
private static long copy(InputStream src, File dstFile) throws IOException {
FileOutputStream dst = new FileOutputStream(dstFile);
try {
byte[] buffer = new byte[1 << 20]; // 1MB
long ret = 0;
int n = 0;
while ((n = src.read(buffer)) >= 0) {
dst.write(buffer, 0, n);
ret += n;
}
return ret;
} finally {
dst.close();
src.close();
}
}
private static File createTemporaryDirectory() {
File baseDirectory = new File(System.getProperty("java.io.tmpdir"));
String directoryName = "jscip-" + System.currentTimeMillis() + "-";
for (int attempt = 0; attempt < 1000; attempt++) {
File temporaryDirectory = new File(baseDirectory, directoryName + attempt);
if (temporaryDirectory.mkdir()) {
return temporaryDirectory;
}
}
throw new IllegalStateException(
"Could not create a temporary directory (tried to make "
+ directoryName
+ "*) to extract TensorFlow native libraries.");
}
}
报错日志
最后是编译时的报错日志，反正能用不研究了：
[ 15%] Linking CXX executable ../../../bin/soplex_c_testing
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/unit_test.dir/TestMain.cpp.o: in function `Catch::Matchers::Floating::WithinUlpsMatcher::describe() const':
TestMain.cpp:(.text+0xc155): undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/unit_test.dir/TestMain.cpp.o: in function `Catch::ReusableStringStream::~ReusableStringStream()':
TestMain.cpp:(.text+0x192c2): undefined reference to `std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::basic_ostringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/unit_test.dir/TestMain.cpp.o: in function `Catch::ReusableStringStream::ReusableStringStream()':
TestMain.cpp:(.text+0x194b1): undefined reference to `std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::basic_ostringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: TestMain.cpp:(.text+0x19553): undefined reference to `std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::basic_ostringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: TestMain.cpp:(.text+0x195c2): undefined reference to `std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::basic_ostringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/unit_test.dir/TestMain.cpp.o: in function `Catch::clara::detail::BasicResult<Catch::clara::detail::ParseResultType> Catch::clara::detail::convertInto<unsigned int>(std::string const&, unsigned int&)':
TestMain.cpp:(.text._ZN5Catch5clara6detail11convertIntoIjEENS1_11BasicResultINS1_15ParseResultTypeEEERKSsRT_[_ZN5Catch5clara6detail11convertIntoIjEENS1_11BasicResultINS1_15ParseResultTypeEEERKSsRT_]+0x20): undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/unit_test.dir/TestMain.cpp.o: in function `Catch::clara::detail::BasicResult<Catch::clara::detail::ParseResultType> Catch::clara::detail::convertInto<long>(std::string const&, long&)':
TestMain.cpp:(.text._ZN5Catch5clara6detail11convertIntoIlEENS1_11BasicResultINS1_15ParseResultTypeEEERKSsRT_[_ZN5Catch5clara6detail11convertIntoIlEENS1_11BasicResultINS1_15ParseResultTypeEEERKSsRT_]+0x20): undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/unit_test.dir/TestMain.cpp.o: in function `Catch::clara::detail::BasicResult<Catch::clara::detail::ParseResultType> Catch::clara::detail::convertInto<double>(std::string const&, double&)':
TestMain.cpp:(.text._ZN5Catch5clara6detail11convertIntoIdEENS1_11BasicResultINS1_15ParseResultTypeEEERKSsRT_[_ZN5Catch5clara6detail11convertIntoIdEENS1_11BasicResultINS1_15ParseResultTypeEEERKSsRT_]+0x20): undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/unit_test.dir/TestMain.cpp.o: in function `Catch::clara::detail::BasicResult<Catch::clara::detail::ParseResultType> Catch::clara::detail::convertInto<int>(std::string const&, int&)':
TestMain.cpp:(.text._ZN5Catch5clara6detail11convertIntoIiEENS1_11BasicResultINS1_15ParseResultTypeEEERKSsRT_[_ZN5Catch5clara6detail11convertIntoIiEENS1_11BasicResultINS1_15ParseResultTypeEEERKSsRT_]+0x20): undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/unit_test.dir/papilo/core/PresolveTest.cpp.o: in function `papilo::ParallelColDetection<double>::execute(papilo::Problem<double> const&, papilo::ProblemUpdate<double> const&, papilo::Num<double> const&, papilo::Reductions<double>&, papilo::Timer const&)':
PresolveTest.cpp:(.text._ZN6papilo20ParallelColDetectionIdE7executeERKNS_7ProblemIdEERKNS_13ProblemUpdateIdEERKNS_3NumIdEERNS_10ReductionsIdEERKNS_5TimerE[_ZN6papilo20ParallelColDetectionIdE7executeERKNS_7ProblemIdEERKNS_13ProblemUpdateIdEERKNS_3NumIdEERNS_10ReductionsIdEERKNS_5TimerE]+0x374): undefined reference to `__cxa_throw_bad_array_new_length'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/unit_test.dir/papilo/core/PresolveTest.cpp.o: in function `papilo::ParallelRowDetection<double>::execute(papilo::Problem<double> const&, papilo::ProblemUpdate<double> const&, papilo::Num<double> const&, papilo::Reductions<double>&, papilo::Timer const&)':
PresolveTest.cpp:(.text._ZN6papilo20ParallelRowDetectionIdE7executeERKNS_7ProblemIdEERKNS_13ProblemUpdateIdEERKNS_3NumIdEERNS_10ReductionsIdEERKNS_5TimerE[_ZN6papilo20ParallelRowDetectionIdE7executeERKNS_7ProblemIdEERKNS_13ProblemUpdateIdEERKNS_3NumIdEERNS_10ReductionsIdEERKNS_5TimerE]+0x5ff): undefined reference to `__cxa_throw_bad_array_new_length'
collect2: error: ld returned 1 exit status
make[2]: *** [papilo/test/CMakeFiles/unit_test.dir/build.make:443: papilo/test/unit_test] Error 1
make[1]: *** [CMakeFiles/Makefile2:1329: papilo/test/CMakeFiles/unit_test.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: ../../../lib/libsoplexshared.so.6.0.2.0: undefined reference to `operator delete(void*, unsigned long)'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: ../../../lib/libsoplexshared.so.6.0.2.0: undefined reference to `std::overflow_error::overflow_error(char const*)'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: ../../../lib/libsoplexshared.so.6.0.2.0: undefined reference to `__cxa_throw_bad_array_new_length'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: ../../../lib/libsoplexshared.so.6.0.2.0: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: ../../../lib/libsoplexshared.so.6.0.2.0: undefined reference to `std::runtime_error::runtime_error(char const*)'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: ../../../lib/libsoplexshared.so.6.0.2.0: undefined reference to `std::invalid_argument::invalid_argument(char const*)'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: ../../../lib/libsoplexshared.so.6.0.2.0: undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: ../../../lib/libsoplexshared.so.6.0.2.0: undefined reference to `operator delete[](void*, unsigned long)'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: ../../../lib/libsoplexshared.so.6.0.2.0: undefined reference to `std::domain_error::domain_error(char const*)'
collect2: error: ld returned 1 exit status
make[2]: *** [soplex/tests/c_interface/CMakeFiles/soplex_c_testing.dir/build.make:112: bin/soplex_c_testing] Error 1
make[1]: *** [CMakeFiles/Makefile2:1553: soplex/tests/c_interface/CMakeFiles/soplex_c_testing.dir/all] Error 2
[ 15%] Building C object scip/src/CMakeFiles/libscip.dir/scip/scipgithash.c.o
[ 15%] Building C object scip/src/CMakeFiles/scip.dir/scip/scipgithash.c.o
[ 15%] Linking CXX shared library ../../lib/libscip.so
[ 15%] Linking CXX executable ../../bin/scip
[ 44%] Built target libscip
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/scip.dir/lpi/lpi_spx2.cpp.o: in function `soplex::SPxBasisBase<double>::readBasis(std::istream&, soplex::NameSet const*, soplex::NameSet const*)':
lpi_spx2.cpp:(.text._ZN6soplex12SPxBasisBaseIdE9readBasisERSiPKNS_7NameSetES5_[_ZN6soplex12SPxBasisBaseIdE9readBasisERSiPKNS_7NameSetES5_]+0x606): undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: lpi_spx2.cpp:(.text._ZN6soplex12SPxBasisBaseIdE9readBasisERSiPKNS_7NameSetES5_[_ZN6soplex12SPxBasisBaseIdE9readBasisERSiPKNS_7NameSetES5_]+0x743): undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()'
/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: CMakeFiles/scip.dir/lpi/lpi_spx2.cpp.o: in function `soplex::SPxSolverBase<double>::factorize()':
lpi_spx2.cpp:(.text._ZN6soplex13SPxSolverBaseIdE9factorizeEv[_ZN6soplex13SPxSolverBaseIdE9factorizeEv]+0x352): undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::basic_stringstream()'
collect2: error: ld returned 1 exit status
make[2]: *** [scip/src/CMakeFiles/scip.dir/build.make:5864: bin/scip] Error 1
make[1]: *** [CMakeFiles/Makefile2:2505: scip/src/CMakeFiles/scip.dir/all] Error 2
^Cmake[2]: *** [soplex/src/CMakeFiles/soplex.dir/build.make:83: soplex/src/CMakeFiles/soplex.dir/soplexmain.cpp.o] Interrupt
make[1]: *** [CMakeFiles/Makefile2:1496: soplex/src/CMakeFiles/soplex.dir/all] Interrupt
make: *** [Makefile:183: all] Interrupt