The GSL now uses a secure encrypted FTP server (FTPS not SFTP) for data transfer. You will receive an email containing your respective login information when the data is available. Please use an FTP client (e.g., FileZilla) or FTP from the command-line (e.g., lftp) that supports explicit FTPS. You may allow for multiple parallel connections to speed up downloading. Basic instructions for FileZilla and command-line, as well as troubleshooting suggestions, are detailed below.
FileZilla Instructions:
1. Opening FileZilla, use the Site Manager to establish a secure connection. Do NOT use the Quickconnect functionality.
2. Fill in the relevant information as shown below substituting your login information in the user/password fields. Press Connect.
3. FileZilla will securely connect to the GSL server and display a directory listing of your files (Remote site pane, right). You can then select all your files (Ctrl+A) and drag them over to a local folder on your computer (Local site panel, left).
4. Downloading progress and success will display in the bottom and top panes respectively.
5. When all downloads have completed. The bottom panel will be empty and you may close FileZilla.
6. If you chose another FTPS compatiable program (WinSCP, Cyberduck), please ensure similar settings.
Command-line instructions using lftp:
1. lftp is a sophisticated command-line utility with many options. You can make a basic connection and log into your data directory using the following statement substituting your username/password from the “data available” email you received.
# … do not include command prompt $
$ lftp -u username,password -e “set ssl:verify-certificate no set ftp:ssl-protect-data true set ftp:ssl-force true” ftp://gslanalyzer.qb3.berkeley.edu:990
2. Assuming no error messages, you will be at the lftp command prompt. Use “help” (without quotes) to see a list of available commands. Common commands are “ls” to list files in your directory as well as “mirror” to retrieve files to your local directory.
# … lftp prompt after successful connection
lftp username@gslanalyzer.qb3.berkeley.edu:/>
# … list files in directory
lftp username@gslanalyzer.qb3.berkeley.edu:/> ls
# … mirror files into local directory
lftp username@gslanalyzer.qb3.berkeley.edu:/> mirror
# … mirror files with continue option (recommended)
lftp username@gslanalyzer.qb3.berkeley.edu:/> mirror -c
# … mirror files with 10 parallel downloads to speed up transfer
lftp username@gslanalyzer.qb3.berkeley.edu:/> mirror -P 10
3. Once the downloads have completed, you may exit lftp using “exit/quit” at the prompt.
# … exit lftp
lftp username@gslanalyzer.qb3.berkeley.edu:/> exit
4. You can combine options in a single line command that skips the lftp prompt and downloads your files into your local directory (ensure standard straight single/double quotes).
# … do not include command prompt $ and password may require quotes
$ lftp -c ‘set ssl:verify-certificate no set ftp:ssl-protect-data true set ftp:ssl-force true; open -u username,password -e “mirror -c; quit” ftp://gslanalyzer.qb3.berkeley.edu:990’
5. If you chose another FTPS compatiable command-line option (wget, curl) please ensure settings for FTPS.
For example using wget (double hyphens may be formatted as em dash):
$ wget -r –secure-protocol=TLSv1_2 –no-check-certificate –no-proxy –ftp-user=username –ftp-password=password -P /path/to/destination/folder ftps://gslanalyzer.qb3.berkeley.edu:990
For curl, recursive downloads or wildcard expansion is not supported as of this writing. Since a filename must be given, you can loop through a directory listing and download all files in this manner:
$ CALL=”curl -k –ftp-ssl –tlsv1.2 –ftp-ssl-reqd –user username:password ftp://gslanalyzer.qb3.berkeley.edu:990″
$ for i in $(eval $CALL | awk -F” ” ‘{print $NF}’); do FILE=$CALL/$i” -O” ; eval $FILE ; done
Troubleshooting:
- Secure FTP connections: Our servers do not support SFTP. Please use an alternative such as FileZilla, lftp, wget, or curl that support FTP through explicit TLS (FTPS).
- Login incorrect: If you are receiving the login incorrect error, your password may have expired or requires regeneration. Please email the GSL and let us know.
- Secure FTP ports: If you are having connection troubles. The connecting/destination machine may need to open TCP ports 12000-12100, though generally this shouldn’t be necessary.
- Command-line errrors: Copy-and-paste from an email or this website may introduce formatting errors. For example, please ensure standard straight single and double quotes, not typographic curly opening/closing quotes when using command-line. Double hyphens can format as longer em dash.
- FileZilla Quickconnect: Connecting to our secure servers via Quickconnect is not supported. Please use the FileZilla Connection Manager as detailed above.
- FileZilla savings passwords error: Ensure that saving of passwords through FileZilla main menu “Edit > Settings > Interface > Passwords” is checked and then retry the connection.