You have data in Redis, but now you need to search it using some basic criteria. The solution is to add a secondary index. So you create some sets, reference the data, and gg wp no re.
But the primary entities have a TTL. Now you can't guarantee your sets are referencing keys that have not expired. Manually checking each key and updating the set is far to expensive to be effective. How do you best keep your sets in sync with the data they represent?
This is an exact scenario I experienced when trying to model Identity Server 4 reference tokens. Access tokens are short lived by design, so the churn could clearly create some very large sets that were mostly filled with bad references.
I needed the access tokens to be indexed by subject ID and client ID, as a user can revoke access tokens by these criteria. I created/updated a sorted set keyed on this data when an access token was created. The TTL of these sorted sets could have been set to the client's access token lifetime and slid as needed. However, the lifetime can technically be altered and I desired to have the data snapshot as the access tokens are issued. So, a TTL of the maximum allowed access token lifetime sufficed. How did I keep the sorted sets synchronized with the actual access token TTLs?
Sorted sets are simply Redis sets that also have a weight, which weight can be used in several useful operations. Remember the topical one, ZREMRANGEBYSCORE, which removes items by weight range. In my case, I assigned the weight of each entry in the sorted set to the Unix timestamp expiration of the relevant access token. Now whenever an access token is directly created or deleted I can simply expire entries out of the associated sorted set with ZREMRANGEBYSCORE.
Admittedly this is not flawless, as it is possible for an access token to slip by if a user does not request a new one for a client and chooses to expire all issued access tokens for that client. The sorted sets are maintained by user activity, so this would be unlikely. And if it does occur, at the very least the sorted set has been historically groomed so it would not contain an unseemly number of de-referenced access tokens. Acceptable losses to have a few keys that don't quite map instead of nearly all of them.