Various reliability fixes #76
Conversation
We want to isolate any sort of replication lag from the DB to make sure that we get any and all attributes and foreign keys after object save. There have been some instances where the version_set is empty.
There was a problem hiding this comment.
I'm not sure this is doing what you think its doing? My understanding is that the local frame instance will already be up to date since you just passed it around to create all these links... But you want something querying the archive later (from the queue) to get up to date data rather than whats on the read replica. Depending on how it queries for the frame data, we could have some fallback where it looks for the frame in the read DB and then if it doesn't find it it tries the write DB before returning a 404? Or just add retry logic in whatever is pulling it off the queue and querying for it since the read db should update itself relatively quickly.
This avoids a query to the version_set, which we've seen issues reading from due to a race condition which has been quite hard to pin down.
BPMs, for instance, have up to hundreds of thousands of related frames. BANZAI queries for these frequently and we don't need related frames in the response, so it's wasteful for us to include them.
archive/frames/models.py
Outdated
| @@ -158,7 +159,10 @@ def as_dict(self, include_thumbnails=False): | |||
|
|
|||
| if self.area: | |||
| ret_dict['area'] = json.loads(self.area.geojson) | |||
| ret_dict['related_frames'] = list(self.related_frames.all().values_list('id', flat=True)) | |||
| if self.configuration_type in settings.EXCLUDE_RELATED_FRAME_TYPES: | |||
There was a problem hiding this comment.
Is this because there are many related frames for BPM? Should we do this for other calibration types too like Bias/Dark/Lampflat? Is this just for efficiency of using less resources on an untargeted /frames/ query?
There was a problem hiding this comment.
Yes, exactly. I don't think there is a need for other frame types, but I've left it configurable for that purpose.
This is on by default, but currently users will be able to request to exclude related frames from their queries. Also update the get_queryset method to avoid prefetching related frames if we know we're not going to use them.
mgdaily
changed the title
If the user has passed ?include_related_frames=false, then do not prefetch related frames, as this is wasteful. Update tests.
…em/science-archive into fix/refresh-frame
?include_related_frames=false. This is to reduce the load on the service and database from repeated queries to frames that have many thousands of related frames that take a long time to fetch and serialize.