Introduction to EHEP tools

Getting started

I am assuming that you have Linux installed on your system. Most of this tutorial will use Bash (not Zsh), so make yourself familiar with some basic Bash commands before going ahead. For Windows users, if you want to stay in the field of EHEP, it is recommended that you shift to a Unix based system (such as linux). If Windows is absolutely necessary for you (e.g., for Photoshop etc), then go for Windows Subsystem in Linux (WSL). Setting up WSL is discussed below. Don't use dual-boot, because it causes frequent problems.

• Official Ubuntu Bash Documentation – Beginner-friendly guide directly from Ubuntu, great for getting started with Bash commands.
• LinuxCommand.org Beginner’s Guide – Offers hands-on exercises and practical examples to actually practice Bash commands.
• Codecademy Command Line Course – Interactive, browser-based learning with exercises and instant feedback.

Installing Miniconda

Miniconda is a tool that helps you install and manage software, especially Python packages, by creating isolated environments. This keeps your main system clean and makes sure everything works smoothly. It’s particularly useful when setting up tools like ROOT because it handles all the necessary dependencies and prevents conflicts with other software.

1. Download Miniconda Installer:
   For Linux, open the terminal and use the following command to download the installer:

   curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

2. Run the Installer:
   After downloading, run the installer by navigating to the download directory and using:

   bash Miniconda3-latest-Linux-x86_64.sh

   Follow the prompts to complete the installation. When terms and conditions show up (which nobody reads!) press Q to skip. Keep the path as default, and type 'yes' when it asks to load on startup. Once done, restart the terminal. If everything is done correctly, your terminal prompt should say (base) in the beginning.

More information on managing miniconda environments can be found here.

Installing ROOT

ROOT is a powerful data analysis framework commonly used in experimental particle physics. It provides essential tools for data processing, statistical analysis, and visualization, making it ideal for handling large datasets from experiments such as those at the Large Hadron Collider. ROOT enables physicists to manage complex data, create visual plots, fit models, and perform advanced statistical analyses, making it a crucial tool in the field of particle physics.

To install ROOT, visit the official ROOT website (build from source). There, you will find detailed instructions on how to build and install for linux. Make sure that you have installed the dependencies first (including git). Or, you can simply go through the following instructions.

• Ubuntu
• ROOT without sudo
• MacOS

sudo apt update && sudo apt upgrade

# required packages:
sudo apt install \
  git binutils cmake dpkg-dev g++ gcc libssl-dev git \
  libx11-dev libxext-dev libxft-dev libxpm-dev python3 \
  libtbb-dev libgif-dev coreutils

# optional packages:
sudo apt install \
  gfortran libpcre3-dev libglu1-mesa-dev libglew-dev libftgl-dev \
  libfftw3-dev libcfitsio-dev libgraphviz-dev libavahi-compat-libdnssd-dev \
  libldap2-dev python3-dev python3-numpy libxml2-dev libkrb5-dev libgsl-dev \
  qtwebengine5-dev nlohmann-json3-dev libmysqlclient-dev libgl2ps-dev \
  liblzma-dev libxxhash-dev liblz4-dev libzstd-dev

cd # make sure that you are in the home area
git clone --branch latest-stable --depth=1 https://github.com/root-project/root.git root_src
mkdir root_build root_install && cd root_build
cmake -DCMAKE_INSTALL_PREFIX=../root_install ../root_src
cmake --build . -- install -j8 #j8/j4 is the number of cores to be used
realpath ../root_install/bin/thisroot.sh #copy this line
    

export PATH="/home/phazarik/root_install/bin/thisroot.sh"

conda install -c conda-forge \
  gcc g++ cmake make expat \
  xorg-libx11 xorg-libxext xorg-libxrender \
  xorg-libxft xorg-libxpm xorg-xproto

# Verify X11 headers
ls $CONDA_PREFIX/include/X11/X.h
ls $CONDA_PREFIX/include/X11/Xfuncproto.h
                

ls $CONDA_PREFIX/include/X11/X.h
ls $CONDA_PREFIX/include/X11/Xfuncproto.h
              
              

# Only activate in interactive shell (otherwise SFTP may fail)
if [[ $- == *i* ]]; then
  export CPATH=$CONDA_PREFIX/include:$CPATH
  export C_INCLUDE_PATH=$CONDA_PREFIX/include:$C_INCLUDE_PATH
  export CPLUS_INCLUDE_PATH=$CONDA_PREFIX/include:$CPLUS_INCLUDE_PATH
  export PKG_CONFIG_PATH=$CONDA_PREFIX/lib/pkgconfig:$PKG_CONFIG_PATH
  export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH
  source ~/root_install/bin/thisroot.sh
fi
              

cmake -S ../root_src -B . \
  -DCMAKE_INSTALL_PREFIX=../root_install \
  -DCMAKE_PREFIX_PATH=$CONDA_PREFIX \
  -Dx11=ON \
  -Dgui=ON \
  -Dasimage=ON \
  -Deve=OFF \
  -Dopengl=OFF \
  -Dgl=OFF \
  -Drgl=OFF \
  -Dbuiltin_glew=OFF \
  -DCMAKE_INCLUDE_PATH=$CONDA_PREFIX/include \
  -DCMAKE_LIBRARY_PATH=$CONDA_PREFIX/lib \
  -Wno-dev
              

cmake --build . --target install -j8

Installing Jupyter Notebook [optional]

Jupyter Notebook is an interactive web application that lets you create and share documents with live code, visualizations, and text (just like Google Colab, except it runs on your local machine). Make sure to install jupyter in the conda environment in which ROOT was installed (base environment from the previous instructions) to ensure its compatibility with ROOT. If it's not base environment, activate the same environment in which ROOT was built.

conda activate base # or any other environment in which ROOT was built 
conda install jupyter
              

Customizing bashrc [optional]

The .bashrc file is a configuration script that runs each time you start a new terminal session in Linux or macOS. It helps set up your shell environment by defining settings, functions, and shortcuts. Be careful while editing this. One useful feature in .bashrc is creating aliases - shortcuts that replace long commands with simpler ones. The following are common aliases that I like. Feel free to copy them and put them in your .bashrc (completely optional).

# Prompt asks for confirmation before running the following:
alias mv='mv -i'
alias cp='cp -i'
alias rm="rm -i"
                
# Running ROOT without the web-browser (with TBrowser):
alias root="root --web=off"
                
# Changing the text and the color displayed at the prompt (for WSL):
PS1='\[\033[0;34m\]custom_name:\w\[\033[0m\]>>'

Windows Subsystem for Linux

Windows Subsystem for Linux (WSL) is a feature in Windows that allows users to run a Linux environment directly on their Windows machine without needing a virtual machine or dual-boot setup. WSL provides a compatibility layer that translates Linux system calls into Windows system calls, with WSL 1 using this translation and WSL-2 running a real Linux kernel in a lightweight virtual machine for better performance and compatibility. It integrates tightly with Windows, allowing access to the Windows file system, interoperability between Linux and Windows applications, and shared networking. This makes it easy for developers and users to run Linux tools and commands alongside their Windows applications seamlessly, without rebooting.

Activating WSL and updating the kernel

To activate Windows Subsystem for Linux (WSL) and update the kernel using the Windows GUI, follow these steps:

1. Enable WSL and Virtual Machine Platform:
   • Open "Control Panel", navigate to "Programs" and then click on "Turn Windows features on or off".
   • In the Windows Features window, scroll down and check the boxes for "Windows Subsystem for Linux" and "Virtual Machine Platform".
   • Click "OK" to apply the changes. Windows will install the required components. You may be prompted to restart your computer.
2. Install a Linux Distribution:
   • Open the Microsoft Store and search for a Linux distribution of your choice (e.g., Ubuntu).
   • Select your preferred distribution from the search results and click on "Install" to download and install it.
3. Update the WSL Kernel (in case the existing kernel is old):
   • Visit the WSL-2 Linux kernel update package webpage.
   • Download the latest WSL-2 Linux kernel update package installer.
   • Run the downloaded installer to update the WSL kernel to the latest version.
   ℹ️ I believe the default version with newer Windows OS is WSL-2 now. For older setup, in case the kernel update is not available in the above link, or the link is down, contact me.
4. Set WSL-2 as the default version (for better performance and compatibility):
   • Search for "Command Prompt" in the Start menu, right-click on it, and select "Run as administrator".
   • In the Command Prompt window, type:

     wsl --set-default-version 2

   • Press Enter to execute the command, which sets WSL-2 as the default version for any new Linux installations.
5. Initialize the user and upgrade the existing packages:
   Once everything above is done, open a new linux terminal by finding the app in your start menu. For the first time, it will ask for a username and password. Just to be safe, pick an username with all small caps and no weird symbols. The password can be as complicated as you want. Note that the password does not appear on the screen while you type, so be careful. Once its done, update the existing packages by running the following commands. It might take a while for the first time.

   
   sudo apt-get update  # finds all the upgradable packages
   sudo apt-get upgrade # downloads and installs all the upgradable packages
                   

Working with Windows terminal

To configure Windows Terminal to open a specific Linux distribution by default, follow these steps:

1. Access Settings in Windows Terminal:
   • Search for "Windows Terminal" in the Start menu and click on it to open.
   • Click on the down arrow (▼) next to the tabs at the top of the window.
   • Select "Settings" from the dropdown menu. This will open the settings UI.
2. Select Default Profile:
   • In the left sidebar, click on "Startup".
   • Under the "Default profile" section, you will see a dropdown menu with a list of available profiles.
   • Select the Linux distribution you want to set as the default from the dropdown menu. This will usually be listed as "Ubuntu-22.04", or the name of your installed distribution.
3. Save Changes:

   After selecting the default profile, the changes are automatically saved. You can close the settings tab. To verify, close and reopen Windows Terminal. It should now open directly to the selected Linux distribution by default.

   You can follow the rest of the instructions using the Windows terminal running linux. To set up your work environment, start with installing Miniconda and follow the rest of the instructions.

A few useful tricks with WSL

Once you are familiar with basic linux commands, add the following useful lines to the .bashrc file.

# Pointing jupyter notebook to the right browser:
alias notebook="BROWSER=/mnt/c/Program\ Files\ \(x86\)/Microsoft/Edge/Application/msedge.exe jupyter notebook --NotebookApp.use_redirect_file=False"

# Opening the present working directory using the Windows file manager:
alias openpwd="/mnt/c/Windows/explorer.exe ."

# Making sure that the duplicate tab opens in the same working directory:
PROMPT_COMMAND=${PROMPT_COMMAND:+"$PROMPT_COMMAND; "}'printf "\e]9;9;%s\e\\" "$(wslpath -w "$PWD")"'
            

CERN Account Management

Once you are registered in the CERN Human Resource database, your Primary CERN Account, CERN e-mail address & mailbox are created automatically. Instructions about the next steps will be emailed there. The following list contains some useful links for account and services management.

• CERN account management
• Resources
• Groups
• TWiki: WorkBook

CERN Grid Certificate

Once, your CERN account is active, you will have to generate a certificate. CERN Grid certificates are digital credentials used to authenticate and securely access CERN's computing resources. You can generate the certificate here. Follow the on-screen instructions and give the certificate file a password. The certificate file is a .p12 file, which expires in a year. It's important to keep this file safe and backed up, as it serves as your digital identity for accessing CERN's grid services.

I am listing the important links related to certificates below.

• CERN certification authority
• Currently available certificates
• CERN computer security course

Adding the certificate to a browser

To integrate the certificate into a web browser, follow these steps:

1. Download the .p12 certificate file to your computer.
2. Go to Settings in your preferred web-browser and search for something like Manage Certificates.
3. Click on the Import button.
4. Select your .p12 file and follow the prompts, including entering the password set during certificate generation.

After importing, the browser will use the certificate for authenticating with CERN services, allowing you to securely access required resources such as CMS-DAS and other internal pages. If the browser says Your connection isn't private .. for cmsweb pages, you can do Advanced > Continue to cmsweb.cern.ch (unsafe) for access. A browser-window will pop-up where you have to select the CERN certificate for processing further.

Adding the certificate to work area

To setup the certificate on the user interface from where you have to work you should follow these steps.

1. Place the p12 certificate file in the .globus directory of your home area. If the directory doesn't exist, create it first. If any old certificate files already exist, remove them.

   
   cd ~
   mkdir .globus
   cd ~/.globus
   mv /path/to/mycert.p12 .
   
   # Remove existing certificates (if any)
   rm -f usercert.pem
   rm -f userkey.pem
                   

2. Use openssl to create the keys, and make them read-only using chmod. This step asks for the password to decrypt the .p12 file. Then, it asks the user to create a new PEMl password [important], which is required to use the certificate while working with the grid.

   
   openssl pkcs12 -in mycert.p12 -clcerts -nokeys -out usercert.pem
   openssl pkcs12 -in mycert.p12 -nocerts -out userkey.pem
   chmod 400 userkey.pem
   chmod 400 usercert.pem
                   

3. Verify if the certificate by running the following.

   voms-proxy-init --rfc --voms cms -valid 192:00

Important links accessible through the CERN account

• Managing EPR points
• CMSSW cross-reference
• All CMS meetings
• All the POG/PAG TWiki pages