Multipart Upload with cURL

Occasionally I find that I need to manually do a multipart upload from the command line. It is always a pain to remember how to craft such a curl command. Here is an example of how to craft a multipart upload for Google Cloud Storage using curl.

First, create a file named something like "multipart.txt" that contains your multipart message. The first line must be the boundary (with -- in front), and the last line must be the boundary with a -- in front and a -- at the end:

--myboundary
Content-Type: application/json; charset=UTF-8

{
  "name": "myObject.txt",
  "contentType": "text/plain"
}

--myboundary
Content-Type: text/plain


This is some text data type.

--myboundary--

Okay, now upload that sucker with curl:

$> curl -X POST -T multipart.txt -H 'Content-Type: multipart/related; boundary=myboundary' 'https://www.googleapis.com/upload/storage/v1/b/bucket_name/o?uploadType=multipart'

Note that we used -T (--upload-file) instead of --data. --data treats data like a web browser would, which is not what we want in this case. We could also have used "--data-binary @multipart.txt" for the same effect, but that's so may more characters.

Uploading to Google Cloud Storage with curl

Sometimes you want to upload a file to Google Cloud Storage. You don't want to use the UI, you don't want to use gsutil, and you don't want to write a program using one of the libraries. You just want to upload it. Manually. With curl.

Authorization is outside the scope of this article. You'll need a way to get an access token with OAuth 2, or you'll need a bucket that allows anonymous object writes.

Here's the command for a basic upload:

$> curl -d 'Contents of the file' 'https://www.googleapis.com/upload/storage/v1beta2/b/mybucket/o?name=name_of_file.txt&uploadType=media'

Bam, easy. But maybe you want to write an actual file and not just some ad-hoc text:

$> curl -X POST -T filename 'https://www.googleapis.com/upload/storage/v1/b/mybucket/o?name=name_of_file.txt&uploadType=media'

This doesn't tell Google Cloud Storage what kind of thing we're uploading, though. We need to set a Content-Type header for that.

$> curl -H "Content-Type: image/jpeg" --data-binary @picture.jpg 'https://www.googleapis.com/upload/storage/v1beta2/b/mybucket/o?name=name_of_file.txt&uploadType=media'