Difference between revisions of "UKCA Chemistry and Aerosol Tutorials: Python notebooks"
(12 intermediate revisions by the same user not shown) | |||
Line 12: | Line 12: | ||
directory using the <tt>pyterm</tt> command to open a terminal that is able to run the necessary Python libraries. |
directory using the <tt>pyterm</tt> command to open a terminal that is able to run the necessary Python libraries. |
||
+ | |||
+ | Copies of these scripts have also been made available on [https://github.com/theabro/ukca/tree/master/Tutorials/UMvn13.0/notebooks GitHub]. |
||
== How to connect == |
== How to connect == |
||
Line 21: | Line 23: | ||
# On your desktop machine, connect to the following web address using a web browser ('''[https://www.mozilla.org/en-GB/firefox/new/ Firefox]''' recommended): |
# On your desktop machine, connect to the following web address using a web browser ('''[https://www.mozilla.org/en-GB/firefox/new/ Firefox]''' recommended): |
||
#* <div style="font-size: 140%;">'''http://localhost:4801/?token=UKCATraining'''</div> |
#* <div style="font-size: 140%;">'''http://localhost:4801/?token=UKCATraining'''</div> |
||
+ | |||
+ | |||
+ | '''Note''' that if you try to connect to the link on your personal computer '''before''' running '''<tt>jab</tt>''' on the VM you will have an error in your browser, even if you have connected properly. |
||
== Technical notes == |
== Technical notes == |
||
Line 38: | Line 43: | ||
[[Image:Mobaxterm_tools.png|400px]] |
[[Image:Mobaxterm_tools.png|400px]] |
||
− | then click ''New SSH tunnel'' in the bottom left corner. Then you will need to set the following information |
+ | then click ''New SSH tunnel'' in the bottom left corner. Then you will need to set the following information, changing the '''<SSH server>''' information to the IP address of your training virtual machine |
[[Image:Mobaxterm_sshtunnel.png|400px]] |
[[Image:Mobaxterm_sshtunnel.png|400px]] |
||
− | and then select your SSH key by clicking the key symbol, navigating to it and selecting it |
+ | and then select your SSH key by clicking the key symbol, navigating to it using the file browser menu that will open, and selecting it |
[[Image:Mobaxterm_sshtunnel_key.png|400px]] |
[[Image:Mobaxterm_sshtunnel_key.png|400px]] |
||
Line 54: | Line 59: | ||
| <Forwarded port> || 4801 |
| <Forwarded port> || 4801 |
||
|- |
|- |
||
− | | <SSH server> || |
+ | | <SSH server> || <tt>[EC2 VM IP ADDRESS]</tt> ''(e.g. 13.40.3.206)'' |
|- |
|- |
||
| <SSH login> || ubuntu |
| <SSH login> || ubuntu |
||
Line 64: | Line 69: | ||
| <Remote port> || 4801 |
| <Remote port> || 4801 |
||
|- |
|- |
||
− | | ''Set SSH key file via key icon on entry in main MobaSSHTunnel panel'' || ''The path to your'' <tt> |
+ | | ''Set SSH key file via key icon on entry in main MobaSSHTunnel panel'' || ''The path to your'' <tt>[KEY NAME]</tt> '' key file (navigate via file manager)'' |
|} |
|} |
||
Line 82: | Line 87: | ||
ssh -N -f -L localhost:4801:localhost:4801 -i /path/to/[KEY NAME] ubuntu@[EC2 VM IP ADDRESS] |
ssh -N -f -L localhost:4801:localhost:4801 -i /path/to/[KEY NAME] ubuntu@[EC2 VM IP ADDRESS] |
||
+ | |||
+ | This connection can be killed using |
||
+ | |||
+ | lsof -ti:4801 | xargs kill -9 |
||
You could also edit your <tt>~/.ssh/config</tt> file to include the following |
You could also edit your <tt>~/.ssh/config</tt> file to include the following |
||
Line 88: | Line 97: | ||
Hostname [EC2 VM IP ADDRESS] |
Hostname [EC2 VM IP ADDRESS] |
||
User ubuntu |
User ubuntu |
||
+ | IdentityFile /path/to/[KEY NAME] |
||
+ | HostKeyAlgorithms ssh-rsa |
||
+ | |||
+ | Host jlab |
||
+ | Hostname [EC2 VM IP ADDRESS] |
||
+ | User ubuntu |
||
+ | ControlMaster auto |
||
+ | ControlPath ~/.ssh/control_sockets%r@%h:%p |
||
IdentityFile /path/to/[KEY NAME] |
IdentityFile /path/to/[KEY NAME] |
||
HostKeyAlgorithms ssh-rsa |
HostKeyAlgorithms ssh-rsa |
||
− | and then connect by |
+ | and then connect for normal work usage by |
− | ssh - |
+ | ssh -X ukca |
+ | |||
+ | and connect to use Jupyer by |
||
+ | |||
+ | ssh -N -f -L localhost:4801:localhost:4801 jlab |
||
You could also make an alias in your <tt>~/.zshrc</tt> or <tt>~/.bashrc</tt>, e.g. |
You could also make an alias in your <tt>~/.zshrc</tt> or <tt>~/.bashrc</tt>, e.g. |
||
− | alias jopen='ssh -N -f -L localhost:4801:localhost:4801 |
+ | alias jopen='ssh -N -f -L localhost:4801:localhost:4801 jlab' |
and then use the command <tt>jopen</tt> from within your terminal. |
and then use the command <tt>jopen</tt> from within your terminal. |
||
Once the connection has been established you'll be able to connect using the link above. |
Once the connection has been established you'll be able to connect using the link above. |
||
+ | |||
+ | To close the connection, type |
||
+ | |||
+ | ssh -O exit -N -f -L localhost:4801:localhost:4801 jlab |
||
=== On the virtual machine === |
=== On the virtual machine === |
||
+ | |||
+ | '''''You will not need to make any changes on the virtual machine - this section is for information only.''''' |
||
The two alias are set in the <tt>~/.bashrc</tt> file on the VM: |
The two alias are set in the <tt>~/.bashrc</tt> file on the VM: |
Latest revision as of 13:01, 31 January 2024
UKCA Chemistry and Aerosol Tutorials at UMvn13.0
During these Tutorials several Jupyter notebooks are provided using both cf-python and the Iris Python libraries. On pre-installed virtual machines these can be found in the
Tutorials/UMvn13.0/notebooks
Due to the SSH connection you will be making to these virtual machines, it is better to run jupyter remotely and then connect to it from the browser on your desktop machine.
You could also use equivalent python scripts held in the
Tutorials/UMvn13.0/scripts
directory using the pyterm command to open a terminal that is able to run the necessary Python libraries.
Copies of these scripts have also been made available on GitHub.
How to connect
- On your VM, change directory using the cd command into the directory that you want to run the scripts from, e.g.
- cd Tutorials/UMvn13.0/notebooks
- On your VM run the command jnotebook to start a jupyter notebook server, or jlab to start a jupyter lab server.
- On your personal computer, open a connection to your AWS instance using one of the methods below.
- On your desktop machine, connect to the following web address using a web browser (Firefox recommended):
Note that if you try to connect to the link on your personal computer before running jab on the VM you will have an error in your browser, even if you have connected properly.
Technical notes
The above instructions make use of some alias and shortcuts to work. The full details of this are:
On your personal computer
Windows via MobaXTerm
On Windows you can use MobaXTerm to connect to the AWS instance to tunnel through to allow you to run Jupyter remotely.
Graphical method
You can use the MobaSSHTunnel (port forwarding) tool to set this up. First, go to the Tools menu and select MobaSSHTunnel (port forwarding)
then click New SSH tunnel in the bottom left corner. Then you will need to set the following information, changing the <SSH server> information to the IP address of your training virtual machine
and then select your SSH key by clicking the key symbol, navigating to it using the file browser menu that will open, and selecting it
Once you have done this you can start and stop the tunnel by clicking the play (triangle) and stop (square) buttons.
Option | Setting |
---|---|
<Forwarded port> | 4801 |
<SSH server> | [EC2 VM IP ADDRESS] (e.g. 13.40.3.206) |
<SSH login> | ubuntu |
<SSH port> | 22 |
<Remote server> | localhost |
<Remote port> | 4801 |
Set SSH key file via key icon on entry in main MobaSSHTunnel panel | The path to your [KEY NAME] key file (navigate via file manager) |
Terminal method
You could also do this by opening a local session, where you run the following command, e.g.
ssh -N -f -L localhost:4801:localhost:4801 -i /drives/x/path/to/[KEY NAME] ubuntu@[EC2 VM IP ADDRESS]
Where /drives/x/path/to/[KEY NAME] is the full path to the location of your private key file that you use to connect to the AWS instance and x is the drive letter, e.g. c for your C: drive.
Once this connection has been established you will be able to connect using the link above. To close this connection, you will need to shut down MobaXTerm.
SSH via a terminal (macOS or GNU/Linux)
You will be able to connect using the following SSH command:
ssh -N -f -L localhost:4801:localhost:4801 -i /path/to/[KEY NAME] ubuntu@[EC2 VM IP ADDRESS]
This connection can be killed using
lsof -ti:4801 | xargs kill -9
You could also edit your ~/.ssh/config file to include the following
Host ukca Hostname [EC2 VM IP ADDRESS] User ubuntu IdentityFile /path/to/[KEY NAME] HostKeyAlgorithms ssh-rsa Host jlab Hostname [EC2 VM IP ADDRESS] User ubuntu ControlMaster auto ControlPath ~/.ssh/control_sockets%r@%h:%p IdentityFile /path/to/[KEY NAME] HostKeyAlgorithms ssh-rsa
and then connect for normal work usage by
ssh -X ukca
and connect to use Jupyer by
ssh -N -f -L localhost:4801:localhost:4801 jlab
You could also make an alias in your ~/.zshrc or ~/.bashrc, e.g.
alias jopen='ssh -N -f -L localhost:4801:localhost:4801 jlab'
and then use the command jopen from within your terminal.
Once the connection has been established you'll be able to connect using the link above.
To close the connection, type
ssh -O exit -N -f -L localhost:4801:localhost:4801 jlab
On the virtual machine
You will not need to make any changes on the virtual machine - this section is for information only.
The two alias are set in the ~/.bashrc file on the VM:
alias jlab="lxterminal -l -e 'export PATH=~/miniconda3/bin:$PATH && jupyter-lab --no-browser --port=4801'" alias jnotebook="lxterminal -l -e 'export PATH=~/miniconda3/bin:$PATH && jupyter notebook --no-browser --port=4801'"
and the
/home/vagrant/.jupyter/jupyter_notebook_config.py
file has the following setting
c.NotebookApp.token = 'UKCATraining'
This means that when you run the command jnotebook or jlab you then launch a new terminal pointing to the python installation in the /home/vagrant/miniconda3/bin directory.
If you have having issues connecting to the Jupyter server over SSH in your local browser and you are connecting via [UKCA_Chemistry_and_Aerosol_Tutorials:_Configuring_X2Go_for_AWS X2Go] then it is possible to open a local Jupyer instance using the Firefox browser installed on the VM. However, it will not be very responsive and might be quite laggy resulting in a poor user experience. It is not recommended to try to open Jupyer in the VM's browser over an SSH connection, as the performance will be even worse.