Implementers howto

[thrawn01]

Create a implementers README with information on how to properly implement and manage a gubernator persistent store.

[valer-cara]

@thrawn01 I'm trying to use gubernator as a library and have some domain-specific logic around it as well as persistence. I've got a few questions to clarify what the intended direction is around library usage.

I see two ways of using gubernator as a library:

1. using the Daemon API and configuring via DaemonConfig.

   • pros
     • handles some of the messier things for you: peer pool lifecycle management; configuration (environ, defaults); ..
     • the cluster operation has test coverage
   • cons
     • http/grpc servers enapsulated, no control over http mux, grpc options, etc..
     • can't set more configuration on V1Instance: Store, Loader, ....;
     • can't configure grpc options like UnaryInterceptor which may be used to wrap additional domain specific logic.
     • for the two issues above, options in DaemonConfig can be added to give more flexibility. That'd be cool, because this high-level Daemon API saves the user a lot of headache. I could take a shot at this if you think it's the right direction.

2. using the V1Instance API

   • pros
     • full configurability of V1Instance: Store, Loader, ....
     • user is free to control their own HTTP/GRPC listeners/muxes, grpc options, etc... no longer encapsulated like in the Daemon API (i'd give this one an extra ⭐)
   • cons
     • user needs to write code that mimics what Daemon does: pool management, configuration (defaults, etc.. that's quite a bunch)
     • code needs to be tested, e2e is no longer covered by the /cluster tests

Ideally I'd use the Daemon API but in a modified version where I have better control on the V1Instance config and where the HTTP/GRPC logic is pluggable and allows me control over grpc opts, http endpoints, etc...

Ideally i'd use V1Instance since it offer most control and composability, but it's a bit difficult to wire up atm to get a working system.

My questions are:

• do you have any hints on hooking up storage from your use of gubernator at mailgun?
• is there a preferred API between Daemon and V1Instance?
• for custom logic, did you use GRPC interceptors, created additional GRPC services, etc..?

[thrawn01]

We took the V1Instance path as it affords us more flexibility as you point out. I don't have any public code available demonstrating the persistence interface, but I can give you a high level.

We have a GubernatorStore struct which implements the gubernator.Store interface, OnChange() looks something like this.

func (g *guberStore) OnChange(r *gubernator.RateLimitReq, item *gubernator.CacheItem) {
	// We don't store ratelimits that will expire in under a minute, only ratelimits with
	// a duration longer than 1 minute are stored.
	if durationUnderMinute(r) {
		return
	}

	gsItem := guberStoreItem{
		ExpireAt:  bsonx.DateTime(item.ExpireAt),
		CacheItem: *item,
		HashKey:   r.HashKey(),
	}

	// Queue the gsItem to be written to the database, we queue writes to the database
	// this so this method doesn't block while waiting for the db to write to return
	g.writer.Queue(&gsItem)
}

We do have custom logic, and for that we created a separate GRPC interface that mimics the gubernator GRPC interface but with our modifications.