Software

From solidwiki
Jump to: navigation, search

introduction

Here is the information hub for the SoLID software for physics study.

Physics event generation is a collection of independent software packages.

Detector simulation is based on Geant4 based simulation framework GEMC http://gemc.jlab.org

The instruction for installation and how to use do event generation and simulationare listed below.

all SoLID related software https://github.com/orgs/JeffersonLab/teams/halla-solid/repositories

Coordinate System

SoLID uses the coordinate system below in lab frame for event generator output, detector arrangement and reconstruction 

electron beam goes along +z axis
The solenoid coil center at the origin (x=0,y=0,z=0),axis along electron beam
y axis is vertical and +y pointing up relative to the ground
x axis is horizontal and +x pointing left when riding the electron beam
+x axis has phi angle 0 deg,  +y axis has phi angle 90 deg
phi angle coverage is from -180 to 180 which can be obtain by atan2(y,x)*180/Pi or TVector3.Phi()*180/Pi

Physics Event Generation

Detector Simulation

SoLID simulation software "solid_gemc" at github https://github.com/JeffersonLab/solid_gemc

See how to run it in container at https://github.com/JeffersonLab/solid_release/blob/master/howto.md

Study

Info

emaillist

proxy

To access outside site from ifarm 

    setenv http_proxy http://jprox.jlab.org:8081 
    setenv https_proxy https://jprox.jlab.org:8081

linux account

SoLID linux account "solid"

Its login is controlled by ssh key which is managed by Ole

linux group

SoLID Linux group is "12gev_solid"

check your account groups by command "groups username" or "id username"

Please ask Ole <ole at jlab.org> to add your jlab account to the group if you want to use SoLID related computing resource at jlab.

Batch Farm Project

slurm job use "-account=halla"

(outdated) auger jobs use "PROJECT: solid"

disk space and access rule

the basics

  • general

Organizing a (small) software project

https://hepsoftwarefoundation.org/training/curriculum.html

https://software-carpentry.org/

  • jlab

2023/05 Software and Computer Workshop 2023

Brad's Farm Use and Computing Resources Tips and Tricks

a list of jlab training

root tip and tricks

https://scicomp.jlab.org/docs/FarmUsersGuide

https://scicomp.jlab.org/docs/JupyterHub

https://cc.jlab.org

important things to remember

  • best practice
    • put source code in github and clone it on home disk for small size or work disk /work/halla/solid/$USER for large size
    • put large output file on volatile /volatile/halla/solid/$USER, decide if you want to keep it, then move it to cache (the move takes no time because the two disk share same backend). Note small files are not good for volatile or cache (both on lustre system), so tar them if have to
    • Before moving files to cache, read "tape use rule" below and ask Ole and Zhiwen for question
  • Is my file safe?
    • file on work disk has backup at /work/XXX/.zfs (old work before fall 2021 has no backup and has failed before)
    • file on home has backup at /home/.snapshot
    • file on group has backup at /group/.snapshot
    • file on volatile has no backup and can be deleted regardless how new it is, if the disk is getting really full which happens often
    • file on mss (tape) has no backup except raw data from real experiment belongs to user "halldata" and tape corruption happens
    • file on cache (write-through) is copied to tape automatically after some days, except for small files below 1MB
    • So use github and keep large file on volatile, then cache

disk info [1] [2] [3] [4]

Disk Space

/home/$USER long term, backup at /home/$USER/.snapshot a few GB your home for private code
/group/solid long term, backup at /group/solid/.snapshot 200GB for shared code, only write to here when you ask SoLID software coordinator
/mss/halla/solid permanent, on tape no limit in practice
/work/halla/solid long term, backup at /work/halla/solid/.zfs/snapshot and it doesn't take up the disk space since 2023 Spring 7.5TB (df can show usage) for small size file, not for large size data
/cache/halla/solid write through cache for mss, short term with auto deletion quota is part of halla for large size file with auto backup to tape
/volatile/halla/solid short term with auto deletion 30TB quota 15TB reserved (df doesn't show correct usage, use the weblink instead at https://scicomp.jlab.org/scicomp/volatileDisk/project) for large size file which you can afford to lose any time

check status under "File System" at https://scicomp.jlab.org (used space is not very up2date, try to run "jvolatile info a-solid" to get latest size)

/cache holds files that are automatically written to tape. Note that files smaller than 1MB are not written to tape to avoid performance problems with the tape system. Small files with important data should be bundled and written to tape periodically using tar or by forcing them to tape with the "jcache put filepath" command.

reference https://scicomp.jlab.org/docs/node/6 and https://scicomp.jlab.org/docs/node/28

Auto deletion rule

On cache and volatile, there are expected to have an expiration date of six months. Also older files will be auto deleted once quota is exceeded.

This is not a problem for cache because all files are backup on tape.

But this quota exceeding auto deletion is very dangerous for volatile because files will be deleted with no warning even when they are not very old to make room refer to [5]

tape use rule

Ask Ole and Zhiwen Zhao for any question or doubt

tape is for long term storage, so DO plan it before use

files can deleted by user owning them, but can only be moved by computer center and dir often stay.

mkdir sure your files below to group 12gev_solid group before backup

jput and jmirror are always safe to use for backup to tape directly right away.

writing to cache will delay backup to tape for some days unless followed by "jcache put". auto back takes some days to happen and some file won't be backup automatically. read this 

tape location /mss/halla/solid/ is organized as following
"backup" for special backup
"sim" for SoLID full simulation
"physics"  and its sub dir for files shared among various physics programs
"subsystem"  and its sub dir for files shared among various subsystems
"user", for any one's personal SoLID related file

use a command like this to find out the size of certain dir on tape, the output is a number in GB
find /mss/halla/solid/backup/ -type f -exec grep size= {} \; | awk -F= '{ a+= $2 } END {print a/1024^3}'

Not all files in tape library are tracked by cacheManager, such as very old, never be jcached files. These files need to remove using jremove. Any files put into tape through cacheManager should use 'jcache tapeRemove'. But after jremove a file from tape, then run 'jcache list /cache/<file-path>', it will figure the file is not in tape and correct its meta data in database.

file ownership

You must belong to the group 12gev_solid for access, ask "Ole at jlab.org" to add you.

To avoid disk quota problems, please make sure that all files that you place in the SoLID directories are owned by Linux group 12gev_solid. If you've already created files that have the wrong group, just change the group ID of your directory and all its files and subdirectories to the correct one. You only should have to do this once:

chgrp -R 12gev_solid your-directory

To make this happen automatically for all newly created files in this directory, set the sgid bit on it:

chmod g+s your-directory

Better yet, set this bit for all subdirectories as well:

find your-directory -type d -exec chmod g+s {} \;

Again, you should only have to do this once.

Additionally, to reduce the chance of file access problems, every time you log in and before starting work in SoLID disk areas, switch your effective group ID to 12gev_solid:

newgrp 12gev_solid

Exit from the shell when done; that will return you to where you were before issuing the newgrp command.


outdated content

  • SoLID GEMC 1.x (simulation in GEANT4 with GEMC 1.x, used for later proposals and pCDR in 2014)
Retrieved from "https://solid.jlab.org/wiki/index.php?title=Software&oldid=2036"