Data Domain Cloud tier : integrating data domain with amazon aws S3

          
          

          
          

          
          

          The following article walks you through the needed steps to configure Data Domain cloud tier capabilities with amazon aws S3

This guide is mainly divided into 4 main parts:
- Adding the required amazon aws user credentials from aws "IAM"
- Importing the CA certificate to enable the communication between Data Domain and S3
- Adding the cloud unit from Data Domain
- Naming of the cloud unit
__First :Adding "IAM" user credentials__
The first step in integrating Data Domain cloud tier with amazon AWS S3 is to add the required AWS user credentials from aws "IAM" . This user credentials will be imported to the data domain system to authorize the communicate with the amazon S3
__The AWS user credentials must have permissions to 
- Create and delete buckets
- Add, modify, and delete files within the buckets they create

**S3FullAccess **is preferred, but these are the minimum requirements :
- CreateBucket
- ListBucket
- DeleteBucket
- ListAllMyBuckets
- GetObject
- PutObject
- DeleteObject
A. Go to httpsaws.amazon.com/
and log in to the AWS console or create a new account if this is your first time:
B. From the top left corner choose services, and search for IAM (AWS Identity and Access Management ), so we can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources:
C. From the IAM page select "users" from the left menu then select " add user":
D. Give your new user a name, for example: "DD_S3_cloudtier" 

Select the access type to give it programmatic access, then click Next:
E. Give this user the required permissions to use S3 resources. Select add user to group , and then select create group:
F. Give a unique name for the group. For example: "S3FullAccess_DD_cloudtier" and then search for "AmazonS3FullAccess". When the option appears in the result menu select it and then click Create group:
G. You will be prompted back to previous menu. Select the group we just created "S3FullAccess_DD_cloudtier" then click Next Tags:
H. On the Review menu, double check that the details you entered are correct then click "Create user ":
__I. we reach an important page
You have now the user "access key ID" and "secret access key". You will use them to integrate the Data Domain with your S3 resources. Click "download .csv" and save this CSV file in a secure place and copy the access key ID and secret access key because we will use them in Data Domain:
__Second: Importing CA certificate__

You must import the CA certificate to enable the communication between your Data Domain system and amazon S3

A. To download the AWS root certificate, go to httpswww.digicert.com/digicert-root-certificates.htm
and select the Baltimore CyberTrust Root certificate:
- If your downloaded certificate has a .CRT extension, it must be converted to a PEM-encoded certificate. If so, use OpenSSL to convert the file from .crt format to .pem. For example, openssl x509 -inform der -in BaltimoreCyberTrustRoot.crt -out BaltimoreCyberTrustRoot.pem

- You can know more about how to convert the certificate to PEM from the following KB article: httpssupport.emc.com/kb/488482
B. Another option is to go to the following page httpsbaltimore-cybertrust-root.chain-demos.digicert.com/info/index.html
and copy the certificate to paste it in the Data Domain system as we will do next:
C. Go to the Data Domain GUI and follow the following procedure:
- 1. Select Data Management > File System > Cloud Units

- 2. In the tool bar, click Manage Certificates. The Manage Certificates for Cloud dialog is displayed

- 3. Click Add

- 4. Select one of these options:
- I want to upload the certificate as a .pem file

Browse to and select the certificate file

- I want to copy and paste the certificate text

Copy the contents of the .pem file to your copy buffer

Paste the buffer into the dialog

We are done with adding the CA certificate. Next we are going to add our S3 cloud unit from Data Domain GUI

__Third: Adding the clout unit to Data Domain__
Here is a quick comparison of some of the differences between DDOS releases and their cloud tier options available:
|DDOS Versio
||Capabilites
|
|6.0
||
|
- Only supports "S3 standart storage" class
- Doesn't have a cloud provider verification method
- Doesn't support large object size feature
|6.1
||
|
- Supports "standard" and "Standard-Infrequent Access (S3 Standard-IA)" storage classes
- 6.1.1.5 >= : Have the cloud provider verification method
- supports large object size feature
|6.2
||
|
- Supports "Standard" , "Standard-IA" and "One Zone-Infrequent Access (S3 One Zone-IA)"
- have the cloud verification method
- supports large object size feature
__from Data domain GUI , follow this procedure to add the S3 cloud unit
- 1. Select Data Management > File System > Cloud Units

- 2. Click Add. The Add Cloud Unit dialog is displayed

- 3. Enter a name for this cloud unit. Only alphanumeric characters are allowed. The remaining fields in the Add Cloud Unit dialog pertain to the cloud provider account

- 4. For Cloud provider, select Amazon Web Services S3 from the drop-down list


- 5. Select the storage class from the drop-down list. Based on the version of the DDOS you will find different options based on the table above

Learn more details about different supported S3 storage classes from the following link to choose the storage class best suitable for your backup needs:
httpsaws.amazon.com/s3/storage-classes/
- 6. Select the appropriate Storage region from the drop-down list

- 7. Enter the provider Access key "as password text", the one we obtained from amazon IAM in step 1

- 8. Enter the provider Secret key "as password text",the one we obtained from amazon IAM in step 1

- 9. Ensure that port 443 (HTTPS) is not blocked in firewalls. Communication with the AWS cloud provider occurs on port 443

- 10. If an HTTP proxy server is required to get around a firewall for this provider, click Configure for HTTP Proxy Server. Enter the proxy hostname, port, user, and password

- 11. if you are having DDOS >= 6.1.1.5 then click the cloud verification button

More details about Data Domain cloud verification tool could be found here: httpssupport.emc.com/kb/521796
If your DDOS version is 6.0 then click add as the cloud verification option is not available in this release

- 12. Click Add. The File System main window now displays summary information for the new cloud unit as well a control for enabling and disabling the cloud unit

-
__Note
You can update the S3 cloud unit access key and secret access key ID afterwards from Data Domain GUI easily if needed

__Third: Naming of the cloud unit__
If we go back now to amazon S3, we will find that the Data Domain system created 3 buckets for this cloud unit:
The naming convention for the 3 buckets are as follows:
- A 16 character hexadecimal string

- A dash character 

- Another 16 character hexadecimal string,
*the hexadecimal string is unique for this cloud unit*

- Another dash character 

- The buckets will end with the string '-d0', '-c0' and '-m0'

- The bucket ending with the string '-d0' is used for data segments

- The bucket ending with the string '-c0' is used for configuration data

- The bucket ending with the string '-m0' is used for metadata

For more details about the naming of the cloud units check the following KB article: httpssupport.emc.com/kb/487833
You are now done with creating S3 cloud unit that is integrated with your Data Domain system, and are ready to start applying data movement policies for your Mtrees to migrate the data to the newly created cloud tier unit .