IBM Spectrum Scale Object Encryption at Rest

Hi Everyone,

I guess you would have reached my blog while searching for help related to setting up file level encryption in Spectrum Scale (formerly GPFS) and I hope you will not be disappointed.

The configuration steps shown here are much similar to the scenario explained in knowledge center. But I just thought to represent the configuration steps in pictorial representation for easy understanding and faster configuration.

As a prerequisite for this setup we would require;

- A RKM (Remote Key Management) sever, currently IBM Security Key Lifecycle Manager (ISKLM) version 2.5.0.1 or later is only supported

- A GPFS cluster which runs Advanced Edition (v4.1)

Once you have both of these ready, you are good enough to configure file level encryption.

(Note: click on the posted screenshots for a full screen view of operations)

Step-1: Just as a validation step, ensure the ISKLM server version is >= 2.5.0.1 (For install and upgrade instructions of ISKLM refer to its product guides)

The ISKLM login via web browser can be made using https://[ISKLM-address]:9080/ibm/SKLM/login.jsp

On the right side top of ISKLM GUI, you can view “About” tab which displays a window showing current version.

Step-2: Create a self signed SSL / KMIP Server certificate
 

 

Step-3: Validate the below entries populated in SKLMConfig.properties, else manually fill in (fips=on is only required if you want to adhere FIPS 140-2 compliance)

[root@isklm ~]# cat /opt/IBM/WebSphere/AppServer/products/sklm/config/SKLMConfig.properties
#Mon Mar 16 20:21:58 IST 2015
KMIPListener.ssl.port=5696
TransportListener.ssl.timeout=10
Audit.handler.file.size=10000
user.gui.init.config=true
config.keystore.name=defaultKeyStore
tklm.encryption.password=AC1CF9EFE025F9F00FDEFA160FDEFA0005DFFB00F123FA16F923F923E9E9
Audit.event.types=runtime,authorization,authorization_terminate,resource_management,key_management
enableKeyRelease=false
TransportListener.tcp.port=3801
Audit.handler.file.name=logs/audit/sklm_audit.log
config.keystore.batchUpdateTimer=60000
Audit.eventQueue.max=0
enableClientCertPush=true
debug=all
TransportListener.tcp.timeout=10
TransportListener.ssl.protocols=SSL_TLS
backup.keycert.before.serving=false
cert.valiDATE=false
config.keystore.batchUpdateSize=10000
maximum.keycert.expiration.period.in.years=50
config.keystore.ssl.certalias=cert1_label
TransportListener.ssl.port=441
fips=on
Audit.event.outcome=success,failure 

Step-4: Restart ISKLM server. Once the restart completes , you should be able to see SSL protocol, KMIP protocol status as “configured”

 

Step-5: Validate whether the created self-signed certificate is configured properly and is in-use

Step-6: Export the configured / in-use server certificate as a file, follow the CLI commands shown below (store this certificate file, it needs to be copied to GPFS nodes)

[root@isklm ~]# cd /opt/IBM/WebSphere/AppServer/bin/
[root@isklm bin]# ./wsadmin.sh -username SKLMAdmin -password Passw0rd -lang jython
WASX7209I: Connected to process "server1" on node SKLMNode using SOAP connector;  The type of process is: UnManagedProcess
WASX7031I: For help, enter: "print Help.help()"
wsadmin>
wsadmin>print AdminTask.tklmCertList('[-alias cert1_label]')
CTGKM0001I Command succeeded.

uuid = CERTIFICATE-c00f107e-6970-44ff-9225-09c86c17dd85
alias = cert1_label
key store name = defaultKeyStore
key state = ACTIVE
issuer name = CN=cert1
subject name = CN=cert1
creation date = 3/16/15 2:02:13 PM India Standard Time
expiration date = 3/15/18 2:02:13 PM India Standard Time
serial number = 657606986627

wsadmin>print AdminTask.tklmCertExport('[-uuid CERTIFICATE-c00f107e-6970-44ff-9225-09c86c17dd85 -format base64 -fileName /root/srvcert]')
CTGKM0001I Command succeeded.
/root/srvcert

Step-7: Create a new device group and choose the Device family as “GPFS”

Step-8: Create Keys associated to the device group created in the above step

You can select the option “Hold new certificate requests pending my approval”, for manual validation of the client.

Copy the keys (for example "KEY-c2cf0496-ded9-42d3-b341-d0ace97fcdcf" is used in step10), and it will be used by GPFS encryption policy 

Step-9: Create a keystore using the ISKLM certificate exported in step-6

[root@eso1 ~]# mkdir /var/mmfs/etc/RKMcerts
[root@eso1 ~]#
[root@eso1 ~]# ls -lrth /root/srvcert
-rw-r--r--. 1 root root 1.1K Mar 16 18:56 /root/srvcert
[root@eso1 ~]#
[root@eso1 ~]# mmauth gencert --cname GPFS_TENANT1 --label client_label --cert /root/srvcert --out /var/mmfs/etc/RKMcerts/ISKLM.p12 --pwd client_label
[root@eso1 ~]#
[root@eso1 ~]# ls -lrth /var/mmfs/etc/RKMcerts/ISKLM.p12
-rw-------. 1 root root 4.0K Mar 17 11:55 /var/mmfs/etc/RKMcerts/ISKLM.p12
[root@eso1 ~]#
[root@eso1 ~]# cat /var/mmfs/etc/RKM.conf
ISKLM_srv {
  type = ISKLM
  kmipServerUri = tls://9.118.46.18:5696
  keyStore = /var/mmfs/etc/RKMcerts/ISKLM.p12
  passphrase = client_label
  clientCertLabel = client_label
  tenantName = GPFS_TENANT1
}

Step-10: Setup an encryption policy and apply it to the file system (device) to which you wanted to enable encryption

[root@eso1 ~]# cat enc_policy
RULE 'p1' SET POOL 'system'
RULE 'Encrypt all files in filesystem with rule E1'
      SET ENCRYPTION 'E1'
      WHERE NAME LIKE '%'
RULE 'simpleEncRule' ENCRYPTION 'E1' IS
      ALGO 'DEFAULTNISTSP800131A'
      KEYS('KEY-c2cf0496-ded9-42d3-b341-d0ace97fcdcf:ISKLM_srv')
[root@eso1 ~]#
[root@eso1 ~]# mmchpolicy gpfs-enc enc_policy -I yes
Validated policy `enc_policy': Parsed 3 policy rules.
Policy `enc_policy' installed and broadcast to all nodes.

Step-11: File I/O failure as the client device is not yet trusted in ISKLM Server

(Reported error - Key could not be fetched)

Step-12: Accept the client device communication certificate

Step-13: File I/O failure as a result of wrong key

Here in this example, I have used a dummy key "KEY-ffc98c44-e8d8-4744-88da-1ca6322c9c4a" which is not associated with the GPFS device group "GPFS_TENANT1".

(Reported Error - Permission denied)

Step-14: File I/O success using correct key and its encryption attributes

(Here gpfs-enc is the filesystem to which we have loaded encryption policy  - step10 and gpfs1 is an unencrypted filesystem)

Step-15: Install and Configure Spectrum Scale Object with the help of  red paper and use gpfs_mount_point as "/gpfs-enc" for object storage path

 

That all, you now have an encryption configured High Performance Scalable Object Storage....