I’m wondering how Thoughtbot and fellow Upcasers use AWS’s users, groups and policies to provide team access to resources such as s3 buckets. I find AWS to be quite overwhelming at times and wonder if this could be a useful topic of a future iteration video.
Here are some things I’d like to know re: your setup.
Can you and your team mates only access the buckets belonging to projects you are working on or do all company members have full read / write access to every bucket?
Do you create users to represent your production and staging servers?
How do you organize your development buckets for any given project?
Do you have one development bucket such as upcase_development which has subfolders for each developer e.g. upcase_development/chris or do you create a separate bucket per developer e.g. upcase_development_chris and upcase_development_ben?
@robodisco I don’t think we have a formal set of best practices here, but here are things I’ve done on applications before that might be helpful:
Configuring AWS configurations isn’t easy, so don’t get too frustrated!
Make sure that production and staging are in separate S3 buckets and that those buckets are accessed by separate access keys (be sure those keys don’t have privileges to create new keys or access each others’ buckets)
I like to use Fog as an abstraction between the app and S3. Fog lets you treat your storage layer in mostly the same way whether it’s on AWS or local, so none of your development machines need access to AWS (they can use a temp folder on disk). A secondary benefit of this strategy is that development and test now don’t require internet access solely due to the use of S3.
I think subfolders for content types is a great idea, but I’d avoid lots of subfolders in development if that approach doesn’t match what you’re doing in production. If you have to use AWS in development, create a separate bucket for each developer.
Re 2. Ahh that was pretty much what i was thinking. Thanks for reassurance.
Re 3. I always thought it was important to mirror production but what you said about fog being an abstraction layer makes sense. Going foward I may just use local storage for development and tests.
Re 4. Yes a seperate bucket for each developer, whilst noisy, seems to be the simplest solution. Though using local storage for dev would make this mute.
On the issue of parity, I think using the same library in production and development (Fog), while it isn’t as good as using the exact same backing storage solution, does at least mean you’re using the same code that you’ve written to handle all environments, which is a step forward for overall parity. This tradeoff won’t always be the right fit, but I think Fog is of a high enough quality that you can depend on them to abstract away implementation details about local vs. AWS storage with confidence.
I think my bias is to always avoid network dependence in development and test environments when I can. Some of my most productive coding has been on trains!