How-To: Install any Python Version on HU-Cluster

You decided to run your latest research software tool not locally on your outdated windows pc but instead on the mighty Humboldt-Universität Berlin Compute Cluster? Good Choice! These powerful machines provide the perfect environment to test, build and develop your latest and greatest tools!

However, your tool is most likely dependent on other software and resources that need to be installed. Unfortunately: Since you are sharing these machines with all other computer science students (and employees), root privileges (to install your dependencies) are only given to a handful of individuals. But your due date is tomorrow, and you really need to run and finish these last experiments.

So what can you do?

Help is here, at least if you depend on different python versions. In this quick tutorial, I show you how you can install, update, and quickly change between your favorite python versions.

[Note] Connecting to the HU Compute Cluster requires an active Humboldt-University account or an account from the computer science department. If you don’t have an account, you might still find the pyenv instructions helpful; thus, you might want to skip the next section.

Connect to the HU-Cluster

With your local Linux or macOS machine, you can access the server via SSH: Open your terminal with the following command:

ssh -l <hu_cs_account> <server>.informatik.hu-berlin.de

where:

1. <hu_cs_account> is the name of your Computer Science Account

2. <server> is the server you want to connect to (Find an overview of all possible servers here))

Alternatively, you can also connect with your general HU account <hu_account> via email:

ssh -l <hu_account>@hu-berlin.de <server>.informatik.hu-berlin.de

You can check the current work load here: Overview.

PyEnv

All we need to do is install pyenv - a simple python version manager tool that allows you to easily switch between multiple versions of python. You can even set local or global system-wide python versions.

Install

1. First, you need to download pyenv:

curl https://pyenv.run | bash

This should automatically install everything along with all dependencies.

1. Set up your shell environment for Pyenv:

Upgrade note: The startup logic and instructions have been updated for simplicity in 2.3.0. The previous, more complicated configuration scheme for 2.0.0-2.2.5 still works.

• Define environment variable PYENV_ROOT to point to the path where Pyenv will store its data. $HOME/.pyenv is the default. If you installed Pyenv via Git checkout, we recommend to set it to the same location as where you cloned it.
• Add the pyenv executable to your PATH if it’s not already there
• run eval "$(pyenv init -)" to install pyenv into your shell as a shell function, enable shims and autocompletion
  • You may run eval "$(pyenv init --path)" instead to just enable shims, without shell integration

The below setup should work for the vast majority of users for common use cases. See Advanced configuration for details and more configuration options.

• For bash:

  Stock Bash startup files vary widely between distributions in which of them source which, under what circumstances, in what order and what additional configuration they perform. As such, the most reliable way to get Pyenv in all environments is to append Pyenv configuration commands to both .bashrc (for interactive shells) and the profile file that Bash would use (for login shells).

  First, add the commands to ~/.bashrc by running the following in your terminal:

  echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
  echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
  echo 'eval "$(pyenv init -)"' >> ~/.bashrc
  

  Then, if you have ~/.profile, ~/.bash_profile or ~/.bash_login, add the commands there as well. If you have none of these, add them to ~/.profile.

  • to add to ~/.profile:

    echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.profile
    echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.profile
    echo 'eval "$(pyenv init -)"' >> ~/.profile
    

  • to add to ~/.bash_profile:

    echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
    echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
    echo 'eval "$(pyenv init -)"' >> ~/.bash_profile
    

• For Zsh:

  echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
  echo 'command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
  echo 'eval "$(pyenv init -)"' >> ~/.zshrc
  

  If you wish to get Pyenv in noninteractive login shells as well, also add the commands to ~/.zprofile or ~/.zlogin.

1. Restart your shell:

exec $SHELL

1. Last but not least, validate your installation:

pyenv --version

Commands:

Now that we have installed the latest version, we can finally install the specific python version with the install command:

pyenv install 3.10.9

To list all already installed versions of python on your system:

pyenv versions

Use the global command to set a specific python version as global (system-wide).

pyenv global 3.10.9
# (Note that you have to install the desired version first with the `install` command)

And to set a specific python version locally (project-based), you can use the local command.

pyenv local 3.10.9

Congratulations!

Congratulations, you did it!

Now you have everything you need! So buckle up, pull that all-nighter, and finish your experiments; it’s about time!