Using Globus Online with LRZ resources
In this tutorial we are showing you how to transfer files to and from LRZ resources using the Globus Online service.
In order to transfer data, you need two GridFTP endpoints. One of them is provided by LRZ on one of the LRZ resources (SuperMUC, Linux Cluster, etc.), the other endpoint is somewhere outside the LRZ. Often supercomputing centers provide these endpoints already and then you can just use them. Or, if you want to exchange data with a machine without GridFTP (e.g., your laptop or desktop), simply install Globus Connect Personal there. In the following tutorial we assume that you will be using Globus Connect Personal.
- You will need a user grid certificate. Follow these instructions in German or these instructions in English if you don't have one. Make sure your certificate's distinguished name (DN) is registered with the ID portal.
- You need a Globus Online account. If you don't have any, please refer to the next section. If you already have one, please remember to select the correct organisation, as suggested by the login page. For users belonging to LRZ and/or its institutional customers, the organisation is usually Globus ID.
- If you want to transfer files to your local machine you will need to setup Globus Connect Personal on your PC. This will enable you to transfer files from your PC to any other endpoint you have access to. If you want to set it up follow the instructions here: https://www.globusonline.org/globus_connect/. You should follow the installation instructions there and setup an endpoint for your PC.
First of all, point your browser to https://www.globusonline.org/ and click on Log in
You are now asked to pick an organisation to authenticate against. Since LRZ is not listed there, please select the "catch all organisation" Globus ID, clicking on the link at the top right corner or using the list (if you start typing the page will restrict the possible choices).
Finally, a login page will appear. Please choose Sign Up from the upper-right corner.
You will be now asked to fill a form in order to create a new account within the Globus ID organisation.
Please remember to accept the Terms of Service, then click on Create ID.
Before we start the transfer we need to make sure our proxy credentials are available to Globus Online so it can authenticate us to the various resources we have available.
- One way to do that is by using MyProxy. You can use the GSISSH-Term tool to do this: Start up the software and choose Tools - MyProxy Tool. Server and port should be preset to
myproxy.lrz.deand 7512. Enter your login credentials and make sure your certificate is available. Set a lifetime that suits your needs; after the time is up, you will have to repeat this step.
- Now make sure your local Globus Connect Personal application is up and running!
Go to the Globus Connect Personal application and click on "Start Transfer". You will be redirected to the file transfer page of the Globus Online website. If you haven't signed in yet you will be asked to do so.
Note: Alternatively you can go to https://www.globusonline.org/, sign in and click on "Start Transfer".
Now we are ready to select our files! On the left pane click on the textbox next to the endpoint tag. You can use there the endpoint name you assigned for your PC while setting up Globus Connect Personal. Wait for the file list to populate and then navigate to the folder where you data resides.
Note: You can use any endpoint you have access to as a source of your transfer as well.
On the right part of the screen choose the preset endpoint for the lrz resource you want to use. Typing 'lrz' will provide you with a list of the LRZ endpoints available. For this example we'll choose "lrz#SuperMUC". After you have selected the endpoint you will be presented with a popup where you can specify your myproxy credential information. As myproxy server use "myproxy.lrz.de" and username and passphrase are those used for uploading your proxy to the myproxy service. You can then click on Authenticate.
Wait for the file list on the right to populate and then navigate to the folder where you want your data stored. Click on the files/folders you want to transfer on the left file list. You will see that the arrow pointing to the right endpoint is activated which means that a transfer can be issued. Clicking on the arrow will start the file transfer. Alternatively, you can select a file in the right pane and click the arrow pointing to the left and thus download the file from LRZ to your laptop.
Note: Optionally you can label the transfer so that you can identify it later using the textbox in the bottom of the page.
After you successfully submit the file transfer you will get a notification on top of the screen with the unique ID of the transfer. The green colour signifies that everything is as it should be.
Click on Activity or RECENT ACTIVITY, you will be taken to a new page where information of all your jobs is listed. A status bar indicates transfers that are going on, a green transfers that have successfully finished while a red X denotes transfers facing a problem (or aborted by the user).
Note: Clicking on the transfer name will provide a popup with more information about the status of the transfer.
Once the transfer is finished, the status bar turns green and you will soon receive an e-mail containing detailed information about the transfer.
With Globus, you can also share big data sets with collegues that do not have access to the LRZ services. For a guide on Globus Sharing, please refer to Globus Sharing.