使用分段上傳來複製物件 - Amazon Simple Storage Service

本文為英文版的機器翻譯版本,如內容有任何歧義或不一致之處,概以英文版為準。

使用分段上傳來複製物件

分段上傳可讓您將物件以一組組件進行複製。本節中的範例示範如何使用分段上傳 API 來複製大於 5 GB 的物件。如需分段上傳的資訊,請參閱在 Amazon S3 中使用分段上傳來上傳和複製物件

您可以透過單一操作來複製小於 5 GB 的物件,而無需使用分段上傳 API。您可以使用 AWS Management Console、 AWS CLI、REST API 或 AWS SDKs物件。如需詳細資訊,請參閱複製、移動和重新命名物件

如需搭配額外檢查總和使用分段上傳來上傳物件的端對端程序,請參閱教學課程:透過分段上傳來上傳物件並驗證其資料完整性

下一節說明如何使用 REST API 或 AWS SDKs物件。

《Amazon Simple Storage Service API 參考》中的下列章節說明了分段上傳的 REST API。若要複製現有的物件,請使用分段上傳 (複製) API,並在要求中新增 x-amz-copy-source 要求標頭以指定來源物件。

您可以使用這些 API 來提出自己的 REST 請求,也可以使用我們所提供的其中一個 SDK。如需搭配 使用分段上傳的詳細資訊 AWS CLI,請參閱 使用 AWS CLI。如需 SDK 的詳細資訊,請參閱「AWS SDK 支援分段上傳」。

若要使用低階 API 複製物件,請執行以下步驟:

  • 呼叫 AmazonS3Client.initiateMultipartUpload() 方法以啟動分段上傳。

  • 儲存 AmazonS3Client.initiateMultipartUpload() 方法傳回之回應物件中的上傳 ID。您所提供的上傳 ID,用於每項分段上傳操作。

  • 複製所有的部分。為了每一個您需要複製的部分,建立一個新的 CopyPartRequest 類別執行個體。提供部分資訊,包含來源和目的地儲存貯體名稱、來源和目標物件金鑰、上傳 ID、部分的第一和最後的位元組位置與部分編號。

  • 儲存 AmazonS3Client.copyPart() 方法呼叫的回應。每一個回應包含 ETag 數值與上傳部分的編號。您需要此資訊才成完成分段上傳。

  • 呼叫 AmazonS3Client.completeMultipartUpload() 方法完成複製操作。

Java

以下範例示範如何使用 Amazon S3 低階 Java API 執行分段複製。如需建立和測試工作範例的說明,請參閱《 適用於 Java 的 AWS SDK 開發人員指南》中的入門

import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.regions.Regions; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.*; import java.io.IOException; import java.util.ArrayList; import java.util.List; public class LowLevelMultipartCopy { public static void main(String[] args) throws IOException { Regions clientRegion = Regions.DEFAULT_REGION; String sourceBucketName = "*** Source bucket name ***"; String sourceObjectKey = "*** Source object key ***"; String destBucketName = "*** Target bucket name ***"; String destObjectKey = "*** Target object key ***"; try { AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .withCredentials(new ProfileCredentialsProvider()) .withRegion(clientRegion) .build(); // Initiate the multipart upload. InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest(destBucketName, destObjectKey); InitiateMultipartUploadResult initResult = s3Client.initiateMultipartUpload(initRequest); // Get the object size to track the end of the copy operation. GetObjectMetadataRequest metadataRequest = new GetObjectMetadataRequest(sourceBucketName, sourceObjectKey); ObjectMetadata metadataResult = s3Client.getObjectMetadata(metadataRequest); long objectSize = metadataResult.getContentLength(); // Copy the object using 5 MB parts. long partSize = 5 * 1024 * 1024; long bytePosition = 0; int partNum = 1; List<CopyPartResult> copyResponses = new ArrayList<CopyPartResult>(); while (bytePosition < objectSize) { // The last part might be smaller than partSize, so check to make sure // that lastByte isn't beyond the end of the object. long lastByte = Math.min(bytePosition + partSize - 1, objectSize - 1); // Copy this part. CopyPartRequest copyRequest = new CopyPartRequest() .withSourceBucketName(sourceBucketName) .withSourceKey(sourceObjectKey) .withDestinationBucketName(destBucketName) .withDestinationKey(destObjectKey) .withUploadId(initResult.getUploadId()) .withFirstByte(bytePosition) .withLastByte(lastByte) .withPartNumber(partNum++); copyResponses.add(s3Client.copyPart(copyRequest)); bytePosition += partSize; } // Complete the upload request to concatenate all uploaded parts and make the // copied object available. CompleteMultipartUploadRequest completeRequest = new CompleteMultipartUploadRequest( destBucketName, destObjectKey, initResult.getUploadId(), getETags(copyResponses)); s3Client.completeMultipartUpload(completeRequest); System.out.println("Multipart copy complete."); } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it, so it returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } } // This is a helper function to construct a list of ETags. private static List<PartETag> getETags(List<CopyPartResult> responses) { List<PartETag> etags = new ArrayList<PartETag>(); for (CopyPartResult response : responses) { etags.add(new PartETag(response.getPartNumber(), response.getETag())); } return etags; } }
.NET

下列 C# 範例示範如何使用 適用於 .NET 的 SDK ,將大於 5 GB 的 Amazon S3 物件從一個來源位置複製到另一個來源位置,例如從一個儲存貯體複製到另一個儲存貯體。若要複製小於 5 GB 的物件,請使用「使用 AWS SDKs」中,所描述的單一操作複製程序。如需 Amazon S3 分段上傳的詳細資訊,請參閱 在 Amazon S3 中使用分段上傳來上傳和複製物件

此範例示範如何使用 適用於 .NET 的 SDK 分段上傳 API,將大於 5 GB 的 Amazon S3 物件從一個 S3 儲存貯體複製到另一個儲存貯體。

using Amazon; using Amazon.S3; using Amazon.S3.Model; using System; using System.Collections.Generic; using System.Threading.Tasks; namespace Amazon.DocSamples.S3 { class CopyObjectUsingMPUapiTest { private const string sourceBucket = "*** provide the name of the bucket with source object ***"; private const string targetBucket = "*** provide the name of the bucket to copy the object to ***"; private const string sourceObjectKey = "*** provide the name of object to copy ***"; private const string targetObjectKey = "*** provide the name of the object copy ***"; // Specify your bucket region (an example region is shown). private static readonly RegionEndpoint bucketRegion = RegionEndpoint.USWest2; private static IAmazonS3 s3Client; public static void Main() { s3Client = new AmazonS3Client(bucketRegion); Console.WriteLine("Copying an object"); MPUCopyObjectAsync().Wait(); } private static async Task MPUCopyObjectAsync() { // Create a list to store the upload part responses. List<UploadPartResponse> uploadResponses = new List<UploadPartResponse>(); List<CopyPartResponse> copyResponses = new List<CopyPartResponse>(); // Setup information required to initiate the multipart upload. InitiateMultipartUploadRequest initiateRequest = new InitiateMultipartUploadRequest { BucketName = targetBucket, Key = targetObjectKey }; // Initiate the upload. InitiateMultipartUploadResponse initResponse = await s3Client.InitiateMultipartUploadAsync(initiateRequest); // Save the upload ID. String uploadId = initResponse.UploadId; try { // Get the size of the object. GetObjectMetadataRequest metadataRequest = new GetObjectMetadataRequest { BucketName = sourceBucket, Key = sourceObjectKey }; GetObjectMetadataResponse metadataResponse = await s3Client.GetObjectMetadataAsync(metadataRequest); long objectSize = metadataResponse.ContentLength; // Length in bytes. // Copy the parts. long partSize = 5 * (long)Math.Pow(2, 20); // Part size is 5 MB. long bytePosition = 0; for (int i = 1; bytePosition < objectSize; i++) { CopyPartRequest copyRequest = new CopyPartRequest { DestinationBucket = targetBucket, DestinationKey = targetObjectKey, SourceBucket = sourceBucket, SourceKey = sourceObjectKey, UploadId = uploadId, FirstByte = bytePosition, LastByte = bytePosition + partSize - 1 >= objectSize ? objectSize - 1 : bytePosition + partSize - 1, PartNumber = i }; copyResponses.Add(await s3Client.CopyPartAsync(copyRequest)); bytePosition += partSize; } // Set up to complete the copy. CompleteMultipartUploadRequest completeRequest = new CompleteMultipartUploadRequest { BucketName = targetBucket, Key = targetObjectKey, UploadId = initResponse.UploadId }; completeRequest.AddPartETags(copyResponses); // Complete the copy. CompleteMultipartUploadResponse completeUploadResponse = await s3Client.CompleteMultipartUploadAsync(completeRequest); } catch (AmazonS3Exception e) { Console.WriteLine("Error encountered on server. Message:'{0}' when writing an object", e.Message); } catch (Exception e) { Console.WriteLine("Unknown encountered on server. Message:'{0}' when writing an object", e.Message); } } } }