Why is there duplicate data in daily payloads
AllRivalBasilisk25 months ago
I have noticed that for each user, I get multiple daily payloads every day and each one contains readings since the beginning of the day. What is the reasoning behind this and what is the best practice in deduplicating this data?
Matthewadmin25 months ago
Hello there
The primary reason why Terra daily, body, nutrition and menstruation payloads contain all readings from the beginning of the day is to account for updates to readings that a provider may perform after the reading is taken, for example: Garmin may update scores after a period of activity. Additionally, this reflects how most data appears users in the app or web-portal provided by the device manufacturer (split by day).
In order to deduplicate Terra payloads:
- Identify the
start_timefound in themetadataobject in each data item and overrite the file or record in your database or bucket, with the new payload (or override if you wish to keep older payloads). If you make a REST request that spans several days, data will be chunked by date (one data object per day) - For
activityandsleeppayloads, deduplication and chunking is different as a sleep/workout session can span across days. Each data item for a sleep/activity payload records one event i.e a nap, sleep, workout, run, exercise, match etc. To deduplicate these, use thesummary_idproperty available in themetadataobject to consolodate the same events.