|
|
| Line 1: |
Line 1: |
| | | | |
| − | = introduction =
| |
| − |
| |
| − | ''''''This page is outdated, see new info at https://github.com/JeffersonLab/solid_release''''''
| |
| − |
| |
| − | jlab ifarm and farm support singularity container. Users can run docker or singularity container on any local machines. singularity can load docker image.
| |
| − |
| |
| − | SoLID docker images are made to run and develope SoLID software to maintain consistent result without need for installation.
| |
| − |
| |
| − | The images are at docker store under account jlabsolid https://store.docker.com/profiles/jlabsolid
| |
| − |
| |
| − | They are in 3 layers:
| |
| − | 1. jlabsolidbase include a centos base images with some necessary rpms;
| |
| − | 2. jlabsolidapps add various applications above jlabsolidbase;
| |
| − | 3. jlabsolid add implementation of SoLID simulation,reconstruction,analysis above jlabsolidapps to complete solid software release
| |
| − |
| |
| − | '''Read [https://hallaweb.jlab.org/wiki/index.php/Note_about_container Note_about_container] for any general things using container'''
| |
| − |
| |
| − | = for SoLID software user =
| |
| − |
| |
| − | '''get jlabsolid image'''
| |
| − | sudo docker pull jlabsolid/jlabsolid:test
| |
| − | location on ifarm, /group/solid/apps/jlabsolid-test.simg
| |
| − | location on web, http://www.phy.duke.edu/~zz81/simg/jlabsolid-test.simg
| |
| − | (jlabsolid-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolid:test" using singularity-2.4.4
| |
| − | with md5sum "2053a0e7c9420a5d60b2e0704c2420f2")
| |
| − |
| |
| − | '''load with singularity on ifarm and farm'''
| |
| − | ssh -X ifarm
| |
| − | module load singularity
| |
| − | echo $DISPLAY (host display is similar to "129.57.70.22:34.0", set it exactly same later inside container)
| |
| − | cd your_work_dir
| |
| − | singularity shell jlabsolid-test.simg
| |
| − | (you are running as yourself in container, not as root)
| |
| − | (your_work_dir at host are mounted in container)
| |
| − | inside container "setenv DISPLAY 129.57.70.22:34.0" or "export DISPLAY=129.57.70.22:34.0" (replace the actual port with what you have)
| |
| − | inside container "xterm" (test you can pass X11 application from inside container
| |
| − |
| |
| − | '''load with singularity on linux host'''
| |
| − | cd your_work_dir
| |
| − | singularity shell jlabsolid-test.simg
| |
| − | (you are running as yourself in container, not as root)
| |
| − | (your_work_dir at host are mounted in container)
| |
| − |
| |
| − | '''load with docker on linux host by using host's Xwindows'''
| |
| − | sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${HOME}:$HOME --ipc=host jlabsolid/jlabsolid:test
| |
| − | (you are running as root in container)
| |
| − | (your home dir is mounted in container)
| |
| − |
| |
| − | '''load with docker on linux host by using container's Xwindows'''
| |
| − | install [https://github.com/mviereck/x11docker x11docker]
| |
| − | wget https://raw.githubusercontent.com/mviereck/x11docker/master/x11docker -O /tmp/x11docker
| |
| − | sudo bash /tmp/x11docker --update
| |
| − | rm /tmp/x11docker
| |
| − | install xorg-x11-server-Xephyr by yum or xserver-xephyr by apt-get
| |
| − | x11docker --desktop --homedir ${HOME} jlabsolid/jlabsolid:test fluxbox
| |
| − | right click on the fluxbox window and open xterm
| |
| − | (you are running as yourself in container, not as root)
| |
| − | (your home dir dir is mounted in container)
| |
| − |
| |
| − | '''run solid_gemc inside container'''
| |
| − | env -i tcsh
| |
| − | source /jlabsolidapps/JLAB_ROOT/setup
| |
| − | setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
| |
| − | setenv SoLID_GEMC /solid_svn/solid_gemc2
| |
| − | setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
| |
| − | cd $SoLID_GEMC/script
| |
| − | solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0 (batch mode)
| |
| − | solid_gemc solid_SIDIS_He3_full.gcard (graphic mode)
| |
| − | exit or ctrl-d (leave container)
| |
| − |
| |
| − | '''How to use container to run jlab farm job'''
| |
| − | examples are at /work/halla/solid/sim/solid_gemc/PVDIS_LD2_JLAB_VERSION_1.3/pass6
| |
| − |
| |
| − | = for SoLID software developer =
| |
| − |
| |
| − | '''checkout solid repo on host and load container'''
| |
| − | cd your_work_dir
| |
| − | svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn (just do it at the first time or run svn up to update it)
| |
| − | cd solid_svn
| |
| − | setenv solid_repo_FULL_PATH $PWD
| |
| − | load with docker
| |
| − | sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:/solid_repo --ipc=host jlabsolid/jlabsolidapps:test
| |
| − | sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v ${PWD}:/solid_repo --ipc=host eictool/jlab_software_2.2 (to run jlab_version=2.2)
| |
| − | or load with singularity
| |
| − | singularity shell --bind ${PWD}:/solid_repo jlabsolidapps-test.simg (download http://www.phy.duke.edu/~zz81/simg/jlabsolidapps-test.simg to run jlab_version=1.3)
| |
| − | singularity shell --bind ${PWD}:/solid_repo eictool_jlab_software_2.2-latest_s2.5.1_20180504.simg (download http://webhome.phy.duke.edu/~zz81/simg/eictool_jlab_software_2.2-latest_s2.5.1_20180504.simg to run jlab_version=2.2)
| |
| − |
| |
| − | '''compile and run solid_gemc inside container'''
| |
| − | tcsh
| |
| − | set prompt = '[%n@%m #Container# %c]$ ' (do this only for singularity to make it clear that you are inside container)
| |
| − | source /solid_repo/set_solid 1.3c (use 1.3c or 2.2c for special setup inside container)
| |
| − | cd $SoLID_GEMC/source/$GEMC_VERSION (just do it at the first time)
| |
| − | scons OPT=1 -j4 (just do it at the first time)
| |
| − | cd $SoLID_GEMC/script
| |
| − | solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0 (batch mode)
| |
| − | solid_gemc solid_SIDIS_He3_full.gcard (graphic mode)
| |
| − | exit or ctrl-d (leave container)
| |
| − |
| |
| − | = for SoLID container developer =
| |
| − |
| |
| − | == work flow ==
| |
| − |
| |
| − | at your local machine with docker installed,
| |
| − | get the right jlabsolidapps image,
| |
| − | then add applications and publish next jlabsolidapps image,
| |
| − | then add solid simulation,reconstruction,analysis code and publish next jlabsolid image
| |
| − | at your local machine use singularity to pull the docker images
| |
| − |
| |
| − | '''The test images are made this way'''
| |
| − | jlabsolidbase:test is auto built on docker cloud from [https://github.com/jlabsolid/container/blob/master/Dockerfile_jlabsolidbase_test this Dockerfile]
| |
| − | jlabsolidapps:test is based on jlabsolidbase:test and build manually by adding jlab framework 1.3 [https://eic.jlab.org/wiki/index.php/Installation_of_framework]
| |
| − | jlabsolid:test is based on jlabsolidapps:test and contained solid svn checkout with compiled solid_gemc
| |
| − |
| |
| − | '''get jlabsolidapps image'''
| |
| − | sudo docker pull jlabsolid/jlabsolidapps:test
| |
| − | location on ifarm, /group/solid/apps/jlabsolidapps-test.simg
| |
| − | location on web, http://www.phy.duke.edu/~zz81/simg/jlabsolidapps-test.simg
| |
| − | (jlabsolidapps-test.simg is build by "sudo singularity pull docker://jlabsolid/jlabsolidapps:test" using singularity-2.4.4 with md5sum "2fa80f6adda35f396ee6bc9c9dd2033b")
| |
| − |
| |
| − | '''load jlabsolidapps with singularity or docker'''
| |
| − | You should checkout svn on host, then load it via "--bind $solid_repo_FULL_PATH:/solid_svn" for singularity, "-v $solid_repo_FULL_PATH:/solid_svn" for docker, "--sharedir $solid_repo_FULL_PATH" for x11docker
| |
| − |
| |
| − | == example ==
| |
| − |
| |
| − | use jlabsolidapps to test application and publish next jlabsolidapps
| |
| − |
| |
| − | use jlabsolidapps to test solid code on host, then push change to solid code repo, then and update /solid_svn, publish next jlabsolid
| |
| − |
| |
| − | <pre>
| |
| − | [user@localhost ~]$ cd $HOME
| |
| − | [user@localhost ~]$ svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn
| |
| − | [user@localhost ~]$ setenv solid_repo_FULL_PATH ${PWD}/solid_svn
| |
| − | [user@localhost ~]$ sudo docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $solid_repo_FULL_PATH:/solid_svn --ipc=host jlabsolid/jlabsolidapps:test
| |
| − | [root@f18d5b96626c /]# tcsh
| |
| − | [root@f18d5b96626c /]# (install application or test solid code)
| |
| − | [root@f18d5b96626c /]# exit
| |
| − | [user@localhost ~]$ sudo docker commit f18d5b96626c
| |
| − | sha256:d55987e37b16b9a17e0795df19b2d6dbe368e0d24891e497805f79a1d733ec80
| |
| − | [user@localhost ~]$ sudo docker image ls
| |
| − | REPOSITORY TAG IMAGE ID CREATED SIZE
| |
| − | <none> <none> d55987e37b16 2 minutes ago 5.62GB
| |
| − | jlabsolid/jlabsolid test ff6d136eee4d 39 minutes ago 6.62GB
| |
| − | jlabsolid/jlabsolidapps test a1b97160252c 11 hours ago 5.62GB
| |
| − | jlabsolid/jlabsolidbase test 5408f7716e45 3 days ago 738MB
| |
| − | [user@localhost ~]$ sudo docker tag d55987e37b16 jlabsolid/jlabsolidapps:test
| |
| − | [user@localhost ~]$ sudo docker image ls
| |
| − | REPOSITORY TAG IMAGE ID CREATED SIZE
| |
| − | jlabsolid/jlabsolidapps test d55987e37b16 2 minutes ago 5.62GB
| |
| − | jlabsolid/jlabsolid test ff6d136eee4d 39 minutes ago 6.62GB
| |
| − | jlabsolid/jlabsolidbase test 5408f7716e45 3 days ago 738MB
| |
| − | [user@localhost ~]$ sudo docker login (login as jlabsolid, ask password from manager)
| |
| − | [user@localhost ~]$ sudo docker push jlabsolid/jlabsolidapps:test (login as jlabsolid, ask password from manager)
| |
| − | [user@localhost ~]$ sudo singularity pull docker://jlabsolid/jlabsolidapps:test
| |
| − | [user@localhost ~]$ md5sum jlabsolidapps-test.simg
| |
| − | e6f21d45b0f58f52631a5c46fa3b4c9d jlabsolidapps-test.simg
| |
| − | </pre>
| |
| − |
| |
| − | '''compile solid_gemc inside container'''
| |
| − | cd $SoLID_GEMC/source/$GEMC_VERSION/
| |
| − | scons OPT=1
| |
| − |
| |
| − | = old =
| |
| − |
| |
| − | (testing as 2017/12)
| |
| − |
| |
| − | jlab ifarm and farm support singularity container. A singularity container is made for the framework
| |
| − | This could be a standard way to run simulation on any machine for consistent result without need for installation.
| |
| − |
| |
| − | Read http://singularity.lbl.gov/faq to understand container and singularity and '''Refer to [https://hallaweb.jlab.org/wiki/index.php/Note_about_singularity Note_about_singularity]'''
| |
| − |
| |
| − | '''singularity image'''
| |
| − | The image is built with singularity-2.3.1, centos7.3 with latest rpm on 20170811, and jlab_version_1.3 framework
| |
| − | location on ifarm for use, /group/solid/apps/singularity_jlab_version_1.3_centos7_20170811.img
| |
| − | location on web, http://www.phy.duke.edu/~zz81/simg/singularity_jlab_version_1.3_centos7_20170811.img.tgz, download it and do "tar zxf" to extract img file
| |
| − |
| |
| − | '''How to use it on ifarm and farm with official svn checkout'''
| |
| − | singularity shell --bind /group/solid/solid_svn:/group/solid/solid_svn /group/solid/apps/singularity_jlab_version_1.3_centos7_20170811.img
| |
| − | source /apps/jlab_root/setup
| |
| − | setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
| |
| − | setenv SoLID_GEMC /group/solid/solid_svn/solid_gemc2
| |
| − | setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
| |
| − | cd $SoLID_GEMC/script
| |
| − | solid_gemc solid_SIDIS_He3_full.gcard (graphic mode, one windows has control and the other shows the detector)
| |
| − | solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0 (batch mode, information shows in terminal)
| |
| − | exit (leave container)
| |
| − |
| |
| − | '''How to use it on any machine with your svn checkout'''
| |
| − | cd (your_choice_of_solid_repo_FULL_PATH)
| |
| − | svn co https://jlabsvn.jlab.org/svnroot/solid solid_svn (checkout the repo)
| |
| − | singularity shell --bind (your_choice_of_solid_repo_FULL_PATH):(your_choice_of_solid_repo_FULL_PATH) container_image_location/singularity_jlab_version_1.3_centos7_20170811.img (enter container. you home dir and your login script are shared between the host and container, sometimes you current dir will also)
| |
| − | source /apps/jlab_root/setup
| |
| − | setenv LD_LIBRARY_PATH ${GEMC}/source:${LD_LIBRARY_PATH}
| |
| − | setenv SoLID_GEMC (your_choice_of_solid_repo_FULL_PATH)/solid_svn/solid_gemc2
| |
| − | cd $SoLID_GEMC/source/$GEMC_VERSION (enter the source dir for solid_gemc)
| |
| − | scons OPT=1 -j4 (compile solid_gemc)
| |
| − | setenv PATH ${SoLID_GEMC}/source/${GEMC_VERSION}:${PATH}
| |
| − | cd $SoLID_GEMC/script
| |
| − | solid_gemc solid_SIDIS_He3_full.gcard (graphic mode, one windows has control and the other shows the detector)
| |
| − | solid_gemc solid_SIDIS_He3_full.gcard -USE_GUI=0 (batch mode, information shows in terminal)
| |
| − | exit (leave container)
| |
| − |
| |
| − | '''How to use it to run jlab farm job'''
| |
| − | examples are at /work/halla/solid/sim/solid_gemc/PVDIS_LD2_JLAB_VERSION_1.3/pass6
| |
