Kopiowanie danych

Please copy data from lustre to project_data in interactive mode on node.

srun --pty /bin/bash

All large operations should not be performed on the access node because the consequences will be felt by all users in the form of slow UI performance

Second thing is that in interactive mode nodes are connected via infiniband network what significantly improves file transfer

This operation can be further accelerated by using rclone utility installed on the cluster, copying data in parallel

Below I send an example of usage:

rclone copy <source path > <target path> --progress --multi-thread-streams=N

During testing, we found that above N=8, there is no longer any additional benefit from faster data transfer