FAQ: Strategies for uploading LARGE files
Posted: Sat Apr 01, 2023 2:32 am
GOTOES gets very heavy usage on a daily basis - some of it from robots and illicit sources. Therefore, there is the need to place limits on how large of a file can be uploaded to the tool. If too many large files are uploaded at once, legitimate users will receive slow page load times and have a poor experience. There are many ways for you to work-around the size limits; I will list the from easiest (and highest fidelity) to hardest (with reduced fidelity):
Strategy 1: ZIP Compress Files
If you are uploading GPX, CSV, or TCX files to the tool, try zipping them. This will make the files MUCH smaller. The tool is most efficient at handling these types of files, so this strategy is your best option - unless you need the extra data contained in FIT files (if so, see next option)...
Strategy 2: Shrink bloated FIT Files
FIT Files tend to be MUCH smaller than GPX or TCX files for the same amount of data. However, FIT files require more processing power. If you are uploading FIT files, and the tool still cannot process them, First try to SHRINK the FIT files here. I have seen problems with certain FIT files. Wahoo files tend to contain a lot of extra junk that sometimes can crash the tool. Also, if you have "Heart Rate Variability" being recorded inside your FIT files, the extra amount of data can be so much, it can crash websites. I have other methods for shrinking FIT files, such as converting them to CSV (makes a larger file but easier to parse and you can zip compress it)... or using the GPS repair tool (repair button appears after you upload). The repair tool removes extra junk... but the shrinker mentioned above should be your first choice.
Strategy 3: Remove Excess Points
A lot of people like to use 1 second recording to get "All the Data". 1 second recording is great for your 5 hour bike rides, but if you're doing an ultra-marathon that takes 24 hours or trying to merge a bike ride across a country, 1 second recording is just too much for my tool (and it is too much for Strava to display their maps correctly). If you're trying to merge files that have just too many points, you can upload them one-at-a-time to the tool and use the "Discard Track Points" Feature, and then re-export the files (see screenshot below). Don't worry: if you have a point every 5 seconds instead of every 1 second, it will still look great on Strava. After you shrink and export all of the files, you can re-import them for a merge and they should process much faster. Remember that you can combine Strategies 1 and 2 with this Strategy 3. I have a discussion on discarding extra trackpoints here
Strategy 4: Multiple Passes
If the techniques above STILL do not work, you can break the files you want to merge up into smaller merges. Let's say you have 30 GPX files you need to merge. If you zip them together, they are still 100Mb. What you can do is zip the FIRST ~10 GPS files TOGETHER and merge those first. When you do the merge, you have two options on how to reduce the export file size:
Strategy 5: Contact Me
If all of these suggestions still fail, you can contact me using the contact form on the main GOTOES site. I can temporarily increase the upload size limit or try and do it for you. FYI, 99% of the time, the first 4 strategies solve the issue for my users, so this should be a last resort option.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Strategy 1: ZIP Compress Files
If you are uploading GPX, CSV, or TCX files to the tool, try zipping them. This will make the files MUCH smaller. The tool is most efficient at handling these types of files, so this strategy is your best option - unless you need the extra data contained in FIT files (if so, see next option)...
Strategy 2: Shrink bloated FIT Files
FIT Files tend to be MUCH smaller than GPX or TCX files for the same amount of data. However, FIT files require more processing power. If you are uploading FIT files, and the tool still cannot process them, First try to SHRINK the FIT files here. I have seen problems with certain FIT files. Wahoo files tend to contain a lot of extra junk that sometimes can crash the tool. Also, if you have "Heart Rate Variability" being recorded inside your FIT files, the extra amount of data can be so much, it can crash websites. I have other methods for shrinking FIT files, such as converting them to CSV (makes a larger file but easier to parse and you can zip compress it)... or using the GPS repair tool (repair button appears after you upload). The repair tool removes extra junk... but the shrinker mentioned above should be your first choice.
Strategy 3: Remove Excess Points
A lot of people like to use 1 second recording to get "All the Data". 1 second recording is great for your 5 hour bike rides, but if you're doing an ultra-marathon that takes 24 hours or trying to merge a bike ride across a country, 1 second recording is just too much for my tool (and it is too much for Strava to display their maps correctly). If you're trying to merge files that have just too many points, you can upload them one-at-a-time to the tool and use the "Discard Track Points" Feature, and then re-export the files (see screenshot below). Don't worry: if you have a point every 5 seconds instead of every 1 second, it will still look great on Strava. After you shrink and export all of the files, you can re-import them for a merge and they should process much faster. Remember that you can combine Strategies 1 and 2 with this Strategy 3. I have a discussion on discarding extra trackpoints here
Strategy 4: Multiple Passes
If the techniques above STILL do not work, you can break the files you want to merge up into smaller merges. Let's say you have 30 GPX files you need to merge. If you zip them together, they are still 100Mb. What you can do is zip the FIRST ~10 GPS files TOGETHER and merge those first. When you do the merge, you have two options on how to reduce the export file size:
- Export as FIT (fit is a smaller overall file size than GPX, CSV or TCX but will keep all of your points)
- Discard Trackpoints (see screenshot below) - and then export as GPX, TCX, or FIT.
Strategy 5: Contact Me
If all of these suggestions still fail, you can contact me using the contact form on the main GOTOES site. I can temporarily increase the upload size limit or try and do it for you. FYI, 99% of the time, the first 4 strategies solve the issue for my users, so this should be a last resort option.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.