provider-boilerplate

Common aspects of providers

The implementation details of the below may vary from provider to provider. Some providers may not perform some steps.

• Receive query event from SNS
• Turn query into a list of possible candidate ids
• Apply weighting to candidate ids
• Iterate over candidate ids to populate full data - usually by querying an external service [1]
• Filter/transform/reduce returned data into a form for consumption by client
• Publish full results over a result stream - this may be one at a time, it may be batched, depending on the provider [1]
• Emit a completion event when no further results are expected.

Notes

[1] Generally a provider which performs a lookup on iteration over the preliminary result set will send results one at a time for speed. Something like an autocomplete provider, which (should probably) perform no such iteration would send all results as a single event.

Additional concerns

• Debugging/logging - possibly want to have common logging messages/formats across providers linked to the steps above.

Proposal

Module will return a middleware pipeline, which can have custom methods inserted into points on the pipeline corresponding to the steps above. Where particular steps are common then reasonable defaults are applied.