Hello everyone, (sorry for this big text)
As many of you know, Concept2 has some issues with data recording. First, the data is recorded in 3-second intervals, and second, it doesn't record "rest" periods properly.
Because of this, I used to - and still do - merge Concept2 data with a GarminIQ app to get data like SmO2, respiration, etc.
However, even after merging, I always have problems when uploading to Intervals.ICU. I can visualize the data, but what happens is that the Power, Speed, and Cadence values during rest periods get "stuck" - they remain stable at the last "work" value. This causes errors in calculating power, load, etc.
I primarily do the merge to correct two values: Distance (which stays at zero) and Time. So if I upload just the FIT file, a 60-minute workout with many rest periods (interval training) ends up showing as only 20 minutes.
Nowadays, for rowing, I use the native Garmin app, which helps somewhat. But for the SkiErg, I still have to use the AppIQ.
I've tried to fix this with Python by locating the laps, selecting the rest laps, and trying to zero out the columns that should be at rest. I have some success with some columns but not others. Also, since it's a CSV file, I lose all the SmO2 data when I convert this CSV file to FIT using Gotoes for uploading to Intervals.ICU.
My question is: Does anyone have any ideas on how I can solve this?
Help with Concept2 files tiemstamp , fileds
Re: Help with Concept2 files tiemstamp , fileds
Hey there -
Can you please upload some test files for me and a work-flow? I have an intervals.icu account and can test this out for you. I'l try and figure out the cause of the issue, and if it is an issue with the GOTOES output, I'll see if I can code in a work-around. You shouldn't be needing to have to code your own solution; the idea behind GOTOES is to be a one-stop solution to GPS issues.
Can you please upload some test files for me and a work-flow? I have an intervals.icu account and can test this out for you. I'l try and figure out the cause of the issue, and if it is an issue with the GOTOES output, I'll see if I can code in a work-around. You shouldn't be needing to have to code your own solution; the idea behind GOTOES is to be a one-stop solution to GPS issues.
Re: Help with Concept2 files tiemstamp , fileds
Apreciatted your effort to help trying to solve this mystery
Row fenix app native.zip
https://drive.google.com/file/d/1CYUTog ... sp=sharing
ROw C2 record.zip
https://drive.google.com/file/d/1aJgqRT ... sp=sharing
skierg forerunner record appIQ.zip"
https://drive.google.com/file/d/1LGh22E ... sp=sharing
skierg C2 record.zip
https://drive.google.com/file/d/1iKZ6Aw ... sp=sharing
Row fenix app native.zip
https://drive.google.com/file/d/1CYUTog ... sp=sharing
ROw C2 record.zip
https://drive.google.com/file/d/1aJgqRT ... sp=sharing
skierg forerunner record appIQ.zip"
https://drive.google.com/file/d/1LGh22E ... sp=sharing
skierg C2 record.zip
https://drive.google.com/file/d/1iKZ6Aw ... sp=sharing
Re: Help with Concept2 files tiemstamp , fileds
I went ahead and examined your files, and it appears that you sent me two pairs of files from different days. I decided to focus on the Garmin/Concept2 merge:
These two files:
20961923596_ACTIVITY.fit
20962072152_ACTIVITY.fit
The first thing I noticed was that they were offset by an hour. That led me to look at the raw data. GOTOES is able to determine the timezone automatically from the Garmin File (it sees and calculates the timezone_offset). On the other hand, the Concept2 file does not have any concept of time zone.
So my first question is: How are you getting these to align for a merge in GOTOES? There are two ways I could imagine resolving this:
1) Because the Concept2 does not record FIT files correctly (there are other fields that are also mis-coded or missing timestamps), I would consider setting the clock on the Concept2 so that at least the files line up with the proper UTC time when you merge them. You might need to adjust this 2x per year if you live in an area that observes daylight savings time.
2) The other way to resolve this would be to upload to the GOTOES time-shift tool.,, but that would require you to know the start time, adding steps...
Anyway, solving the alignment appears to be only part of the problem. The next issue is the "frozen" rest periods. I investigated your Concept2 file, and it seems as if this isn't an issue of recording every 3 seconds (we could easily fix it if that were the case)... instead, it seems that the Concept 2 completely stops recording during rest periods:
In my screenshot above, you can see between line 263 and 265, there is a gap of over 3 minutes with no recording... THIIS is where the issue is happening.... Cocept2 stops recording data points when you stop.
A "boneheaded" solution might be to pull very lightly during the rest (not sure if it would work with your routine, but instead of just stopping entirely, maybe you could continue to row at a greatly diminished pace. This could potentially show "intervals" without having to resort to trickery.
I did also try to play around with your original file by doing some pre-processing on it using GOTOES' "Try to Insert Stopped Time" feature. Basically, I took your Concept2 file, and time-shifted it by one hour. Then when the settings page showed up, I checked the "Try to Insert Stopped Time" and put the speed smoothing relatively low (but not all the way to zero).
When you click that option, GOTOES will add "activity" start and stops whenever you're missing more than 30 seconds worth of data. Then, you can take the pre-processed file, export it, and merge it with whatever Garmin file you have containing the SPO2 etc. fields.
By the way, you don't have to lose the Moxy data fields when using CSV. If you still continue to use CSV files for editing, just take the completed CSV file and upload it to the merge tool in GOTOES alongside the Garmin files that has the Moxy developer data fields. Then, for the Garmin device that has the Moxy Fields, check the "Connect IQ" box which will add them to the list of streams being exported by GOTOES. Then be sure to UNCHECK any of the fields that you worked on in the CSV file from the Garmin file... That way, you will only include the distance, Power, Cadence, etc from the CSV file.. and still get the Moxy data fields from the Garmin file - without the unedited Garmin data interfering.. Let me know if that makes sense...
These two files:
20961923596_ACTIVITY.fit
20962072152_ACTIVITY.fit
The first thing I noticed was that they were offset by an hour. That led me to look at the raw data. GOTOES is able to determine the timezone automatically from the Garmin File (it sees and calculates the timezone_offset). On the other hand, the Concept2 file does not have any concept of time zone.
So my first question is: How are you getting these to align for a merge in GOTOES? There are two ways I could imagine resolving this:
1) Because the Concept2 does not record FIT files correctly (there are other fields that are also mis-coded or missing timestamps), I would consider setting the clock on the Concept2 so that at least the files line up with the proper UTC time when you merge them. You might need to adjust this 2x per year if you live in an area that observes daylight savings time.
2) The other way to resolve this would be to upload to the GOTOES time-shift tool.,, but that would require you to know the start time, adding steps...
Anyway, solving the alignment appears to be only part of the problem. The next issue is the "frozen" rest periods. I investigated your Concept2 file, and it seems as if this isn't an issue of recording every 3 seconds (we could easily fix it if that were the case)... instead, it seems that the Concept 2 completely stops recording during rest periods:
A "boneheaded" solution might be to pull very lightly during the rest (not sure if it would work with your routine, but instead of just stopping entirely, maybe you could continue to row at a greatly diminished pace. This could potentially show "intervals" without having to resort to trickery.
I did also try to play around with your original file by doing some pre-processing on it using GOTOES' "Try to Insert Stopped Time" feature. Basically, I took your Concept2 file, and time-shifted it by one hour. Then when the settings page showed up, I checked the "Try to Insert Stopped Time" and put the speed smoothing relatively low (but not all the way to zero).
- Try and Insert Stopped Time
By the way, you don't have to lose the Moxy data fields when using CSV. If you still continue to use CSV files for editing, just take the completed CSV file and upload it to the merge tool in GOTOES alongside the Garmin files that has the Moxy developer data fields. Then, for the Garmin device that has the Moxy Fields, check the "Connect IQ" box which will add them to the list of streams being exported by GOTOES. Then be sure to UNCHECK any of the fields that you worked on in the CSV file from the Garmin file... That way, you will only include the distance, Power, Cadence, etc from the CSV file.. and still get the Moxy data fields from the Garmin file - without the unedited Garmin data interfering.. Let me know if that makes sense...
- You can combine an edited CSV file with your original Garmin File to preserve the Connect IQ fields.
Re: Help with Concept2 files tiemstamp , fileds
SKi time shift apllied - https://drive.google.com/file/d/1z_9BB3 ... sp=sharing
Ski C2 https://drive.google.com/file/d/1swLv-B ... sp=sharing
Ski Garmin - https://drive.google.com/file/d/1LVq-tx ... sp=sharing
In this case, you can see that even by doing this, the time difference between the two files increased significantly for some reason.I believe with this i dont need to use feature "Try to Insert Stopped Time" , correct?
3. I also tried using the CSV file and merging it with the FIT file through the Import Queue to avoid losing data of MOxy . However, unlike when I merge two FIT files, the option to merge doesn't appear when I use a CSV file.
Ski C2 https://drive.google.com/file/d/1swLv-B ... sp=sharing
Ski Garmin - https://drive.google.com/file/d/1LVq-tx ... sp=sharing
Sorry for sending the wrong files, and thank you for analyzing them. Yes, with the SkiErg, it sometimes happens that the files have different timestamps. I use the TimeShift feature in GoldenCheetah to try to align them (I still haven't figured out why this occurs, even after I check the time on the Erg itself).The first thing I noticed was that they were offset by an hour. That led me to look at the raw data. GOTOES is able to determine the timezone automatically from the Garmin File (it sees and calculates the timezone_offset). On the other hand, the Concept2 file does not have any concept of time zone.
So my first question is: How are you getting these to align for a merge in GOTOES? There are two ways I could imagine resolving this:
1) Because the Concept2 does not record FIT files correctly (there are other fields that are also mis-coded or missing timestamps), I would consider setting the clock on the Concept2 so that at least the files line up with the proper UTC time when you merge them. You might need to adjust this 2x per year if you live in an area that observes daylight savings time.
Regarding the attached SkiErg files, this time I tried not programming rest periods and instead left it as a completely new interval without making manual adjustments. To some extent, it seems to have worked for Power and Pace—interestingly, it did not work for Cadence. (The TimeShift also needs to be applied here).A "boneheaded" solution might be to pull very lightly during the rest (not sure if it would work with your routine, but instead of just stopping entirely, maybe you could continue to row at a greatly diminished pace. This could potentially show "intervals" without having to resort to trickery.
In this case, you can see that even by doing this, the time difference between the two files increased significantly for some reason.I believe with this i dont need to use feature "Try to Insert Stopped Time" , correct?
3. I also tried using the CSV file and merging it with the FIT file through the Import Queue to avoid losing data of MOxy . However, unlike when I merge two FIT files, the option to merge doesn't appear when I use a CSV file.
Re: Help with Concept2 files tiemstamp , fileds
What do you mean when you say, "unlike when I merge two FIT files, the option to merge doesn't appear when I use a CSV file."? Are you uploading to the GOTOES GPS file merge tool? If so, there isn't really an "option to merge," as the primary intent of that tool is to... merge files.... In other words, that is exactly what the tool does when you use it - merge all loaded files. I'll take a look at the rest of your reply later (I'm catching up on a backlog of inquiries, but wanted to briefly settle some glaring questions first.)
Re: Help with Concept2 files tiemstamp , fileds
Unlike when I merge two FIT files, the "Merge" option does not appear when I try to use a CSV file.
What I meant is: when I try to merge files in Merge GOTOES — for example, a Garmin .fit file and a .CSV file from an IQ app or another source — the usual data selection options are not available.
As for timeshift, the issue seems to only happen with the SkiErg data. I don’t think it's a problem with the GOTOES tool itself, so I use timeshift to correct it every time (possibly a bug in C2 software).
Also, after trying workouts without including "Rest" and just using a normal interval format, for some reason the activity now shows a much shorter moving time than before.
What I meant is: when I try to merge files in Merge GOTOES — for example, a Garmin .fit file and a .CSV file from an IQ app or another source — the usual data selection options are not available.
As for timeshift, the issue seems to only happen with the SkiErg data. I don’t think it's a problem with the GOTOES tool itself, so I use timeshift to correct it every time (possibly a bug in C2 software).
Also, after trying workouts without including "Rest" and just using a normal interval format, for some reason the activity now shows a much shorter moving time than before.
Re: Help with Concept2 files tiemstamp , fileds
Hi!
For your first issue, can you please provide a screenshot?
For the second issue, did you see the “try and insert stopped time” checkbox? You can check that, and if you want to adjust the stopped time, just fiddle with the “minimum speed to consider moving”. Please note that only FIT supports “stopped time” - neither GPX nor TCX support moving/stopped time. When you see stopped time on one of these file types, whatever viewer you are using needs to apply its own algorithm to calculate the moving or stopped time. Gotoes gives you control over moving and stopped time by letting you adjust the minimum speed to consider moving.
For your first issue, can you please provide a screenshot?
For the second issue, did you see the “try and insert stopped time” checkbox? You can check that, and if you want to adjust the stopped time, just fiddle with the “minimum speed to consider moving”. Please note that only FIT supports “stopped time” - neither GPX nor TCX support moving/stopped time. When you see stopped time on one of these file types, whatever viewer you are using needs to apply its own algorithm to calculate the moving or stopped time. Gotoes gives you control over moving and stopped time by letting you adjust the minimum speed to consider moving.