Getting 504 Gateway Time-out when downloading data in a browser
Downloading datasets sometimes results in a 504 Gateway Time-out error. We are in the process of working through this with our developers - what you are experiencing is due to the ever increasing size of the data causing timeouts.
In the example below, downloading an mps dataset using OGC Web Services WFS request ....resulted in a 504 Gateway Time-out.
https://hub.mapstand.com/gs/ows?service=WFS&version=1.0.0&request=GetFeature&typename=mps%3Amps_mapping_pipeline&outputFormat=json&srs=EPSG%3A4326&format_options=charset%3AUTF-8&apikey=************
This was resolved by replacing gs with geoserver (see below)
https://hub.mapstand.com/geoserver/ows?service=WFS&version=1.0.0&request=GetFeature&typename=mps%3Amps_mapping_pipeline&outputFormat=json&srs=EPSG%3A4326&format_options=charset%3AUTF-8&apikey=************
Accessing the data using the geoserver url i.e https://hub.mapstand.com/geoserver/wfs and using the API Access Token will give a user access to all the layers available on the app – but it doesn’t know anything about the geonode (HUB) and therefore will only return the production (mps) datasets. This is a higher performance way of getting the data as the geonode proxy slows things down and you may get timeouts.
The gs proxy server (https://hub.mapstand.com/gs/wfs ) acts like a tunnel between geoserver and geonode (the HUB) and manages the permissions locally. Therefore, If you want to access the development (phoenix) datasets you must use the geonode proxy url (gs) it will return both production and development datasets.
WFS is not ideal for large datasets and we are fast tracking a workaround which will deliver daily updated geopackage downloads from a bucket. In the meantime we recommend you use geoserver and API access token in the URL for the production (mps) datasets and the gs with API access token for the development datasets (phoenix).
Another solution is to use the Hub to download the data in geopackage format. You may still "exceed the maximum HARD allowed size". If this is the case, try using the download filtered dataset option.
1. Open the dataset you wish to download
2. Select Filter from menu and create a filter using the attribute or Area of Interest (remember to apply and save)
3. Select Download > dataset from the menu. To download only the filtered portion, make sure the Download filtered dataset toggle is switched ON. Set up the download window as below and click on the Export button.
4. Repeat step 2. above using another filter - see below for example ....
5. Repeat step 3 to export that portion of the data
6. Repeat steps 2 and 3 until all portions of the dataset have been exported.