Installing SNAP and Snappy

SNAP and snappy installation instructions

[This content is replicated on the file system at: /apps/jasmin/supported/snap/INSTALLATION/README-snap-on-jasmin]

These step by step instructions will enable you to create an updated version of snap / snappy on JASMIN when required.

A fully scripted install has not been provided. This is partly because the interactive installer uses a GUI, and partly because of the need to do things as different users and on different machines, but the advantage is that the only commands run as root are to create writable directories and then fix up ownership, and the installer itself, which does potentially unknown things, is then run under a normal username without write access to the OS -- this prevents it causing inconsistencies between sci machines.

It might be possible to use the terminal-based equivalent of the installer (unset the DISPLAY variable) to script this, but the questions which it asks might not be completely repeatable, so it would probably require an "expect" script - probably not worth the effort given how often it would be run.

Alan Iwi 20/8/2021

SNAP installation

  • Go to a clean temporary directory

  • get the interactive installer 

wget https://step.esa.int/downloads/8.0/installers/esa-snap_all_unix_8_0.sh

(a copy of this is also in the directory that contains the README file: /apps/jasmin/supported/snap/INSTALLATION)

  • As root, create a writable directory for the new installation You will run the install as non-root and then fix up the permissions afterwards.
# mkdir /apps/jasmin/supported/snap/8.0 # chown /apps/jasmin/supported/snap/8.0
  • Move any ~/.snap directory out of the way from your home directory (if you have it)
mv ~/.snap ~/OLD.snap

  • Run the installer *** AS YOUR USERNAME NOT AS ROOT. ***

umask 022 # ensure files will be created with sensible permissions so we will 
          # only need to change the ownership
sh esa-snap_all_unix_8_0.sh     # DO NOT DO THIS AS ROOT.

[NOTE - if preferred you can use terminal-based installer if you unset the DISPLAY variable when doing this i.e.: env DISPLAY="" ; sh esa-snap_all_unix_8_0.sh ]

  • Answer the prompts as follows:

    • accept agreement
    • if asked, say yes to deleting all user snap data [not sure where it finds this, I have tried very hard to find out where it is looking and what it is referring to, but if you say no then when you run the desktop it seems to attempt some Netbeans update, not sure why]
    • tell it destination folder [e.g. /apps/jasmin/supported/snap/8.0]
    • if asked, confirm okay to install to existing directory [this is empty one you just created]
    • install all components
    • do not create symlinks [module file will be used for that instead]
    • don't configure for use with python [you will do this afterwards from the command line]
    • say no to running snap desktop

  • As root, take back all of the files that were installed. 

# chown -R root.root /apps/jasmin/supported/snap/8.0
  • As your user, run the app (using its full path for now) and check that it starts okay
/apps/jasmin/supported/snap/8.0/bin/snap
  • If the RPM mesa-dri-drivers is not installed on the local machine, then you will get a bunch of warnings on the terminal output and you will also get a window saying "Unable to initialize WWWorldMapToolView" but apart from that, the application ought to open and run.

SNAPPY installation:

Building jpy wheel

During this installation, you will need a "wheel" file for jpy with a filename such as jpy-0.10.0.dev1-cp37-cp37m-linux_x86_64.whl.

This should be built for the correct python version but apart from that, it is not specific to any particular snappy installation, and if you are providing a later version of snappy it is okay to copy the one from the previous version, otherwise, you will need to build it as follows. If you don't need to build it, skip down to "configuring snappy".

The build will need to be on a machine that has the JDK of a version that matches the version of JRE on the sci machines (the sci machines don't have JDK, but on a sci machine do "rpm -qa | grep openjdk" to see the JRE version). To avoid installing software on the sci machines, sci2-test was used for this purpose, and JDK was installed (as root) using:

# yum install java-1.8.0-openjdk-devel.x86_64

Then (as non-root on the development machine in question), build as follows:

First, you create a conda env and install maven into it - this is because Jaspy doesn't have Maven, which is needed for the build, so the aim is to create a conda environment that contains the same version of Python as is in Jaspy and also contains maven.

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh 
sh Miniconda3-latest-Linux-x86_64.sh 
PATH=/path/to/my_conda_base/bin/:$PATH    # substitute path based on wherever you just put it

conda create -n jpy_build python=3.7.9=hffdb5ce_0_cpython # this based on "conda list" inside jaspy 
source activate jpy_build 
conda install maven

Then the jpy wheel build itself - you will need to set the JAVA_HOME environment variable to point to the jdk environment (if unsure do e.g. "rpm -ql java-1.8.0-openjdk-devel" to see where the files live).

You will have to make sure that you are using the correct version of python, so note the module load jaspy in the instructions here.

git clone https://github.com/bcdev/jpy.git
cd jpy
module load jaspy/3.7/r20210320  # or whichever jaspy version you want to use with snappy
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/ 
python setup.py bdist_wheel

This will produce a wheel file in the dist directory. If you are working under a local /tmp directory then copy it somewhere that you'll have access on the sci machines (e.g. your home directory)

cp dist/jpy-0.10.0.dev1-cp37-cp37m-linux_x86_64.whl ~/

Configuring snappy

[This is done on an ordinary sci machine, so if you used a development machine to build the jpy wheel, then log out of that. You probably won't have write permission on /apps.]

You will be running snappy-conf, telling it where to find python and where to put snappy.

First of all, as root, create a writable directory for yourself, for example as follows. (The versioning here is based on the SNAP version and the JASPY module version. snappy is provided as part of SNAP so doesn't seem to have an independent version number of its own.)

  # mkdir -p /apps/jasmin/supported/snap/snappy/8.0_jaspy-3.7-r20210320
  # chown <your_username> <path_to_directory_you_just_made>

Now in a NON-ROOT shell, do the following (adapt version numbers as required):

snap_dir=/apps/jasmin/supported/snap/8.0
snappy_dir=/apps/jasmin/supported/snap/snappy/8.0_jaspy-3.7-r20210320

module load jaspy/3.7/r20210320

mkdir -p $snappy_dir/snappy/
wheel_file=~/jpy-0.10.0.dev1-cp37-cp37m-linux_x86_64.whl   # or wherever you have it
cp $wheel_file $snappy_dir/snappy/

# now you are ready to run snappy-conf
python=$(which python)
echo $python  # check that this is from Jaspy

$snap_dir/bin/snappy-conf $python $snappy_dir

It should say "Configuring SNAP-Python interface..." After it says "Done." it does not immediately return to the command prompt but should do within less than a minute.

# should have some files installed under here...
ls -lR $snappy_dir/

As root, now take back the files that you created...

# chown -R root.root /apps/jasmin/supported/snap/snappy/8.0_jaspy-3.7-r20210320

SNAPPY testing

Use the test_snappy.py file in the same directory as this readme.

Make a copy and change snappy_path variable near the top.

Then do:

module load jaspy <br>python /path/to/test_snappy_copy.py

It will set the python path to where you selected and also find under there a test data file which is provided as part of the distribution. The test will try to import the module, read the test data file and check some values.

You might get warnings about GDAL versions but it should get to "test completed", e.g.:

    testing SNAPPY import...
    INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Incompatible GDAL 3.1.4 found on system. Internal GDAL 3.0.0 from distribution will be used.
    INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
    INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
    INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
    testing SNAPPY file read...
    INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Internal GDAL 3.0.0 set to be used by SNAP.
    INFO: org.hsqldb.persist.Logger: dataFileCache open start
    test completed

Providing module files

Separate module files for snap and snappy are provided. The snap one does not depend on a particular version of Jaspy, but the snappy one does so loads the relevant Jaspy and snap modules as well as setting up the PYTHONPATH variable

Using root access, copy the previous module files from under:

/apps/jasmin/modulefiles/snap/ 
# and
/apps/jasmin/modulefiles/snappy/

to the new version numbers in the filenames and also edit the version numbers inside these.

Testing module files

Do "module load" on the new module files. Type "snap" (or at least, "which snap") - check that it can be found without supplying a full path.

Also edit the test_snappy program that you used above, to comment out the sys.path.append line and check that it will still run (i.e. PYTHONPATH set up correctly).

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.

Still need help? Contact Us Contact Us