S3 Backend for Afero

About

It provides an afero filesystem implementation of an S3 backend.

This was created to provide a backend to the ftpserver but can definitely be used in any other code.

I'm very opened to any improvement through issues or pull-request that might lead to a better implementation or even better testing.

Key points

• Download & upload file streaming
• 75% coverage (all APIs are tested, but not all errors are reproduced)
• Very carefully linted

Known limitations

• File appending / seeking for write is not supported because S3 doesn't support it, it could be simulated by rewriting entire files.
• Chtimes is not supported because S3 doesn't support it, it could be simulated through metadata.
• Chmod support is very limited

How to use

Note: Errors handling is skipped for brevity but you definitely have to handle it.

import(
	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/credentials"
	"github.com/aws/aws-sdk-go/aws/session"
  
	s3 "github.com/fclairamb/afero-s3"
)

func main() {
  // You create a session
  sess, _ := session.NewSession(&aws.Config{
    Region:      aws.String(region),
    Credentials: credentials.NewStaticCredentials(keyID, secretAccessKey, ""),
  })

  // Initialize the file system
  s3Fs := s3.NewFs(bucket, sess)

  // And do your thing
  file, _ := fs.OpenFile("file.txt", os.O_WRONLY, 0777)
  file.WriteString("Hello world !")
  file.Close()
}

Thanks

The initial code (which was massively rewritten) comes from:

• wreulicke's fork
• Itself forked from aviau's fork.
• Initially proposed as an afero PR by rgarcia and updated by aviau.