Difference between revisions of "Software"
(→linux group) |
(→The following content is outdated as of 2019) |
||
| Line 212: | Line 212: | ||
| − | = | + | = outdated content = |
| − | + | * [[Software_Detector_Simulation| simulation using svn]] (how to run do simulation with svn, outdated as 2019) | |
| − | + | * [[Software_2019| software using svn]] (how to run solid software with svn, outdated as 2019) | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
* [[sim geant4| SoLID GEMC 1.x]] (simulation in GEANT4 with GEMC 1.x, used for later proposals and pCDR in 2014) | * [[sim geant4| SoLID GEMC 1.x]] (simulation in GEANT4 with GEMC 1.x, used for later proposals and pCDR in 2014) | ||
* [[sim geant3| SoLID Comgeant]] (simulation in GEANT3 used for the early proposals) | * [[sim geant3| SoLID Comgeant]] (simulation in GEANT3 used for the early proposals) | ||
Revision as of 11:56, 15 October 2024
Contents
introduction
Here is the information hub for the SoLID software for physics study.
Physics event generation is a collection of independent software packages.
Detector simulation is based on Geant4 based simulation framework GEMC http://gemc.jlab.org
The instruction for installation and how to use do event generation and simulationare listed below.
all SoLID related software https://github.com/orgs/JeffersonLab/teams/halla-solid/repositories
Coordinate System
SoLID uses the coordinate system below in lab frame for event generator output, detector arrangement and reconstruction
electron beam goes along +z axis The solenoid coil center at the origin (x=0,y=0,z=0),axis along electron beam y axis is vertical and +y pointing up relative to the ground x axis is horizontal and +x pointing left when riding the electron beam +x axis has phi angle 0 deg, +y axis has phi angle 90 deg phi angle coverage is from -180 to 180 which can be obtain by atan2(y,x)*180/Pi or TVector3.Phi()*180/Pi
Physics Event Generation
Detector Simulation
SoLID simulation software "solid_gemc" at github https://github.com/JeffersonLab/solid_gemc
It is distributed by container at https://github.com/JeffersonLab/solid_release
outdated simulation page as of 2019 Software Detector Simulation
Study
Info
emaillist
- mailing list solid_software@jlab.org (registration and archive)
proxy
To access outside site from ifarm
setenv http_proxy http://jprox.jlab.org:8081 setenv https_proxy https://jprox.jlab.org:8081
linux account
SoLID linux account "solid"
Its login is controlled by ssh key which is managed by Ole
linux group
SoLID Linux group is "12gev_solid"
check your account groups by command "groups username" or "id username"
Please ask Ole <ole at jlab.org> to add your jlab account to the group if you want to use SoLID related computing resource at jlab.
Batch Farm Project
slurm job use "-account=halla"
(outdated) auger jobs use "PROJECT: solid"
disk space and access rule
the basics
- general
Organizing a (small) software project
https://hepsoftwarefoundation.org/training/curriculum.html
https://software-carpentry.org/
- jlab
2023/05 Software and Computer Workshop 2023
Brad's Farm Use and Computing Resources Tips and Tricks
https://scicomp.jlab.org/docs/FarmUsersGuide
https://scicomp.jlab.org/docs/JupyterHub
important things to remember
- best practice
- put source code in github and clone it on home disk for small size or work disk /work/halla/solid/$USER for large size
- put large output file on volatile /volatile/halla/solid/$USER, decide if you want to keep it, then move it to cache (the move takes no time because the two disk share same backend). Note small files are not good for volatile or cache (both on lustre system), so tar them if have to
- Before moving files to cache, read "tape use rule" below and ask Ole and Zhiwen for question
- Is my file safe?
- file on work disk has backup at /work/XXX/.zfs (old work before fall 2021 has no backup and has failed before)
- file on home has backup at /home/.snapshot
- file on group has backup at /group/.snapshot
- file on volatile has no backup and can be deleted regardless how new it is, if the disk is getting really full which happens often
- file on mss (tape) has no backup except raw data from real experiment belongs to user "halldata" and tape corruption happens
- file on cache (write-through) is copied to tape automatically after some days, except for small files below 1MB
- So use github and keep large file on volatile, then cache
Disk Space
| /home/$USER | long term, backup at /home/$USER/.snapshot | a few GB | your home for private code |
| /group/solid | long term, backup at /group/solid/.snapshot | 200GB | for shared code, only write to here when you ask SoLID software coordinator |
| /mss/halla/solid | permanent, on tape | no limit in practice | |
| /work/halla/solid | long term, backup at /work/halla/solid/.zfs/snapshot and it doesn't take up the disk space since 2023 Spring | 7.5TB (df can show usage) | for small size file, not for large size data |
| /cache/halla/solid | write through cache for mss, short term with auto deletion | quota is part of halla | for large size file with auto backup to tape |
| /volatile/halla/solid | short term with auto deletion | 30TB quota 15TB reserved (df doesn't show correct usage, use the weblink instead at https://scicomp.jlab.org/scicomp/volatileDisk/project) | for large size file which you can afford to lose any time |
check status under "File System" at https://scicomp.jlab.org (used space is not very up2date, try to run "jvolatile info a-solid" to get latest size)
/cache holds files that are automatically written to tape. Note that files smaller than 1MB are not written to tape to avoid performance problems with the tape system. Small files with important data should be bundled and written to tape periodically using tar or by forcing them to tape with the "jcache put filepath" command.
reference https://scicomp.jlab.org/docs/node/6 and https://scicomp.jlab.org/docs/node/28
Auto deletion rule
On cache and volatile, there are expected to have an expiration date of six months. Also older files will be auto deleted once quota is exceeded.
This is not a problem for cache because all files are backup on tape.
But this quota exceeding auto deletion is very dangerous for volatile because files will be deleted with no warning even when they are not very old to make room refer to [5]
tape use rule
Ask Ole and Zhiwen Zhao for any question or doubt
tape is for long term storage, so DO plan it before use
files can deleted by user owning them, but can only be moved by computer center and dir often stay.
mkdir sure your files below to group 12gev_solid group before backup
jput and jmirror are always safe to use for backup to tape directly right away.
writing to cache will delay backup to tape for some days unless followed by "jcache put". auto back takes some days to happen and some file won't be backup automatically. read this
tape location /mss/halla/solid/ is organized as following "backup" for special backup "sim" for SoLID full simulation "physics" and its sub dir for files shared among various physics programs "subsystem" and its sub dir for files shared among various subsystems "user", for any one's personal SoLID related file
use a command like this to find out the size of certain dir on tape, the output is a number in GB
find /mss/halla/solid/backup/ -type f -exec grep size= {} \; | awk -F= '{ a+= $2 } END {print a/1024^3}'
Not all files in tape library are tracked by cacheManager, such as very old, never be jcached files. These files need to remove using jremove. Any files put into tape through cacheManager should use 'jcache tapeRemove'. But after jremove a file from tape, then run 'jcache list /cache/<file-path>', it will figure the file is not in tape and correct its meta data in database.
file ownership
You must belong to the group 12gev_solid for access, ask "Ole at jlab.org" to add you.
To avoid disk quota problems, please make sure that all files that you place in the SoLID directories are owned by Linux group 12gev_solid. If you've already created files that have the wrong group, just change the group ID of your directory and all its files and subdirectories to the correct one. You only should have to do this once:
chgrp -R 12gev_solid your-directory
To make this happen automatically for all newly created files in this directory, set the sgid bit on it:
chmod g+s your-directory
Better yet, set this bit for all subdirectories as well:
find your-directory -type d -exec chmod g+s {} \;
Again, you should only have to do this once.
Additionally, to reduce the chance of file access problems, every time you log in and before starting work in SoLID disk areas, switch your effective group ID to 12gev_solid:
newgrp 12gev_solid
Exit from the shell when done; that will return you to where you were before issuing the newgrp command.
outdated content
- simulation using svn (how to run do simulation with svn, outdated as 2019)
- software using svn (how to run solid software with svn, outdated as 2019)
- SoLID GEMC 1.x (simulation in GEANT4 with GEMC 1.x, used for later proposals and pCDR in 2014)
- SoLID Comgeant (simulation in GEANT3 used for the early proposals)
