Well - I got my hands on a limited preview of AWS Kinesis!  It promises to  be one of the most insane realtime data-grepping vacuums ever built.  Remains to write, I was very excited to tackle it this afternoon (got my welcome email last night).

After I plowed through the morning's chores, I made a coffee, read the docs -- and then Googled around to see if any low-level libraries were available; I knew inside that I was in for the library build given our needs, but I was hopeful to avoid building the signature.  Quickly found a piece I'd met before, the AWS SDK for PHP and its Signature V4 class. This thing is rigged to be configurable, and despite my intuition, I loaded it up and ran a few tests.  It was far too slow.  That old adage about performance vs. configurability always holds doesn't it. The abstractions and options might be great for multipurpose usage, but my Kinesis usage is scheduled to fire several thousand times a second - the overhead on the SDK is simply too great.

Back to the drawing board then, I built a test from scratch that used cURL - with success (after some hacking at it admittedly, the API error messages are a bit cryptic at times).  I thought others could benefit from this, and so I post it here!  


(I put a Gist up for now, sorry to make you jump through hoops)

I'll follow up with our real world experience on the Kinesis