Information on storage migration for Master Users

Dear Master User,

This notification concerns you in your role as master user of a Linux Cluster project at LRZ.

As part of the baseline LRZ services, NAS shares (HOME-directories, and maybe also a PROJECT area) are currently assigned to Linux Cluster projects. Because the NAS systems used for the Linux Clusters are nearing the end of their serviceability period (Note: other NAS shares are not necessarily impacted), the data stored on these NAS shares will need to be migrated to the next-generation long-term storage systems, called Data Science Storage (DSS). In conjunction with this migration, LRZ is also redefining the policies and workflows for use of long-term data storage; this means that master users will be also designated the role of data curators, with management rights and obligations for appropriately assigned subsets of storage systems ("Containers").

LRZ will provide successor containers for the HOME and PROJECT areas that closely mimic the current setup, to minimize the initial effort on your part (Note: if you already are managing or using DSS containers, these are unrelated to the storage areas discussed here). In particular, the current project quotas will be transferred to DSS containers that contain user-specific subdirectories with access rights to individual user accounts. The path name of the PROJECT directory will be supplied to each user; for the HOME directories, the path names are stored in the LRZ identity management system (which supplies the HPC systems with the necessary information); it will then be incumbent on each user to

  1. store away the PROJECT path name in their system prologues,
  2. migrate data from the legacy PROJECT area to the new DSS area, and
  3. review job scripts and configuration files for necessary adjustments to path names

The maximum baseline service quota for a Linux project (at no extra cost) will be 10 TBytes. If the project's needs exceed this capacity, please ask for a quote for paid-up DSS services (at least an additional 20 TByte).

Because the information about DSS assignments will be sent around by e-mail, it will be necessary that a valid mail address is stored in the LRZ Identity Management System (IDM) for all users; since only Master Users can add this information to the IDM, you will be contacted by your users in those (few) cases where an entry is still missing; to assist your users, you might want to proactively review and update the accounts in your project.

The HOME area will usually be migrated to a dedicated DSS container by LRZ, with some delay.

Timetable for migration to DSS

DateActivityAction on part ofComments

 

NAS PROJECT provisioning ends LRZ Responsibility for adding DSS project areas and inviting users is transferred to data curator. Note however that LRZ will provide a default setup for the Cluster PROJECT area, so there is no need to provision this area on your part, unless you are explicitly contacted by (see "special cases" below).

 

User notification of DSS PROJECT link LRZ by e-Mail to all users that currently have a PROJECT area.

-  

Linux Cluster maintenance LRZ New storage setup will be configured during this period.
The cluster systems will be unavailable for user operation during this period.
DSS PROJECT available on HPC systems;
New HOME path set in IDM
LRZ NAS PROJECT path is mounted read-only.

-  

Migration of PROJECT data users Old data path is temporarily available via the PROJECT_LEGACY variable.

 

HOME is set to new DSS value LRZ Data in HOME directory are migrated by LRZ, but users may need to adjust path names. Old data path is temporarily available via the HOME_LEGACY variable.

From  

End of service for NAS systems LRZ NAS paths will be taken offline. Please contact the Service Desk for outstanding data migration issues.

Special cases

In case your project has special requirements that already fall outside the baseline LRZ services, you will receive additional information on necessary migration actions from LRZ in due time.


Verfasser: R. Bader
veröffentlicht: 2019-08-21