3

So I have a Django app hosted on Heroku that has an endpoint which accepts image uploads. The images are saved to my AWS bucket, which is charged to me. Right now, I am able to upload images by making a multipart request with the Postman app.

My goal is to allow my android app to accept image uploads from the user. However, my understanding of web applications is very limited, and I don't understand how to prevent people from just uploading arbitrarily large files to my endpoint and racking up a big charge to my AWS/Heroku accounts. After all, it's just a URL I send POST requests to. So,

  • What is a good way of preventing people from uploading indiscriminately to my server so I dont wake up with a $10k bill from Amazon?

    • My blind guess is that there is some practice of sending authentication from the client - Is there a good way of doing this without making users go through a log in service?

  • If I mess up, will hosting services, (like Heroku or Amazon) provide some safety net to prevent racking up massive charges?

  • What do big services like imgur do to prevent people from uploading huge amounts of data?

Eric S.
  • 207
  • 1
  • 6
  • Looks related: http://programmers.stackexchange.com/questions/145704/ – 9000 Aug 31 '15 at 17:12
  • I removed the resource request from the question as that is off-topic. There are also [many existing questions](http://programmers.stackexchange.com/search?q=rest+secure) on the topic (not sure if there is an exact duplicate). –  Aug 31 '15 at 17:14
  • Besides authentication, you can also think of measures like limiting the file size, accepting only a limited set of mime types in the POST, checking that the uploaded file content matches with the specified mime type and/or extension, etc. – Bart van Ingen Schenau Aug 31 '15 at 17:19
  • Is the Android app uploading directly to Amazon or does it upload to your webserver and then you upload to Amazon? – The Muffin Man Aug 31 '15 at 17:44
  • @TheMuffinMan Er, I think it's the former case; I save it only in my Amazon bucket, but the post request goes through my Django framework. – Eric S. Aug 31 '15 at 17:46
  • So it sounds like the file is upload to your server with Django on it (uploaded not saved) and it is in memory, you then persist it by sending it to Amazon. – The Muffin Man Aug 31 '15 at 18:43

2 Answers2

3

If your data is only meant to be persisted in Amazon S3 it's not performant to proxy it through your web server and should be avoided if possible.

What you can do is create a POST policy that contains constraints on what the uploader (your Android app can do).

You might create the following policy:

  • Can only upload to specific bucket
  • Specify min/max size of file
  • etc

The beauty of this is that your web server can dynamically generate these policies on the fly and send them to the client (Android app). Perhaps your user pays for a specific amount of storage space, that would be possible to control using this method.

You can read about Post policy here: http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html

Your workflow might go something like this:

  1. User navigates to upload view in your app
  2. App makes request to your rest service to retrieve policy
  3. Rest service constructs policy JSON, base64 encodes it and sends it back
  4. App sends data to Amazon S3 directly passing along the policy
  5. Amazon will read the policy, make sure it hasn't been tampered with and accepts or rejects the file based on the constraints of the policy.

There's also this library for Android https://mobile.awsblog.com/post/Tx1V588RKX5XPQB/TransferManager-for-Android but you have to provide your Amazon S3 credentials which I don't think is very secure.

The Muffin Man
  • 423
  • 3
  • 12
0

You can try to limit size and number of files the user can send to your API and use an strategy to identify the application (or device) when you receiving the request something like a key in the request header (for example 'Authentication: key '). This way you difficult people to use a REST client to post many files.

Eldius
  • 111
  • 2