This is a draft document that was built and uploaded automatically. It may document beta software and be incomplete or even incorrect. Use this document at your own risk.

Jump to content Jump to page navigation: previous page [access key p]/next page [access key n]

Administration Guide / Documentation Updates

Applies to SUSE Enterprise Storage 5.5 (SES 5 & SES 5.5)

D Documentation Updates #

This chapter lists content changes for this document since the initial release of SUSE Enterprise Storage 4. You can find changes related to the cluster deployment that apply to previous versions in https://documentation.suse.com/ses/5.5/single-html/ses-deployment/#ap-deploy-docupdate.

The document was updated on the following dates:

D.1 The Latest Documentation Update #

Bugfixes #

Added hint on verifying profile proposal in Section 1.5, “Adding an OSD Disk to a Node” (https://bugzilla.suse.com/show_bug.cgi?id=1134736).
Remove minion's key from the Salt master temporarily when problematic in Section 1.3, “Removing and Reinstalling Cluster Nodes” (https://bugzilla.suse.com/show_bug.cgi?id=1120662).
Added procedure on making ceph.conf parameters unique in Section 1.12, “Adjusting ceph.conf with Custom Settings” (https://bugzilla.suse.com/show_bug.cgi?id=1116349).
Extended a tip on preventing rebalancing after adding a node in Section 1.1, “Adding New Cluster Nodes” (https://bugzilla.suse.com/show_bug.cgi?id=1131044).
Added performance notes to Book “Deployment Guide”, Chapter 13 “Exporting Ceph Data via Samba” and Chapter 16, NFS Ganesha: Export Ceph Data via NFS (https://bugzilla.suse.com/show_bug.cgi?id=1124674).
Added Section 7.1.1.5, “Migrating from a Legacy SSD Rule to Device Classes” (https://bugzilla.suse.com/show_bug.cgi?id=1112883).
Added a list of corresponding software repositories in Section 1.10, “Updating the Cluster Nodes” (https://bugzilla.suse.com/show_bug.cgi?id=1117474).
Added Section 9.5, “Advanced Features” (https://bugzilla.suse.com/show_bug.cgi?id=1120706).
Extended examples of Ceph services identification in Section 3.1.3, “Identifying Individual Services” (https://bugzilla.suse.com/show_bug.cgi?id=1120682).
Updated Section 11.6, “Setting Up an Example Tiered Storage” to use device classes (https://bugzilla.suse.com/show_bug.cgi?id=1114827).
Added example to Book “Deployment Guide”, Chapter 7 “Customizing the Default Configuration”, Section 7.1.5 “Updates and Reboots during Stage 0” (https://bugzilla.suse.com/show_bug.cgi?id=1119451).
Use multiple internal time sources in Section 20.4, “Time Synchronization of Nodes” (https://bugzilla.suse.com/show_bug.cgi?id=1119571).
Added listing of pool snapshots in Section 8.4, “Pool Snapshots” (https://bugzilla.suse.com/show_bug.cgi?id=1113911).
Added Section 7.1.1, “Device Classes” (https://bugzilla.suse.com/show_bug.cgi?id=1113292).
Added tcp_nodelay explanation in Section 13.4, “Configuration Parameters” (https://bugzilla.suse.com/show_bug.cgi?id=1106274).
Added more details on pool compression in Section 8.5, “Data Compression” (https://bugzilla.suse.com/show_bug.cgi?id=1113934).
Improved Section 8.3.2, “Migrate Using Cache Tier” and added Section 8.3.3, “Migrating RBD Images” (https://bugzilla.suse.com/show_bug.cgi?id=1113900).
Added Important: Removed OSD ID Still Present in grains (https://bugzilla.suse.com/show_bug.cgi?id=1107464).

D.2 October, 2018 (Documentation Maintenance Update) #

General Updates #

Moved Section 20.1, “Identifying Orphaned Partitions” into Chapter 20, Hints and Tips.
Extended Section 17.4.3, “Managing RADOS Block Devices (RBDs)”, mainly added a section about snapshots (Fate #325642).
Added Section 8.3, “Pool Migration” (Fate#322006).
Inserted Section 3.2, “Restarting Ceph Services using DeepSea” into Chapter 3, Operating Ceph Services.
Added Section 1.7, “Replacing an OSD Disk”.

Bugfixes #

Extended Section 1.7.2, “Automated Configuration” (https://bugzilla.suse.com/show_bug.cgi?id=1111442).
Added /etc/ceph to the list of backup content in Book “Deployment Guide”, Chapter 6 “Backing Up the Cluster Configuration”, Section 6.1 “Back Up Ceph COnfiguration” (https://bugzilla.suse.com/show_bug.cgi?id=1153342).
Removed 'default-update-no-reboot' as it is 'default' now, in Book “Deployment Guide”, Chapter 7 “Customizing the Default Configuration”, Section 7.2 “Modifying Discovered Configuration” (https://bugzilla.suse.com/show_bug.cgi?id=1111318).
Fixed path to *-replace file in Section 1.7, “Replacing an OSD Disk” (https://bugzilla.suse.com/show_bug.cgi?id=1111470). Removed AppArmor snippets from Book “Deployment Guide”, Chapter 5 “Upgrading from Previous Releases” and added Section 1.13, “Enabling AppArmor Profiles” (https://bugzilla.suse.com/show_bug.cgi?id=1110861).
Added Chapter 5, Monitoring and Alerting (https://bugzilla.suse.com/show_bug.cgi?id=1107833).
Disable kernel NFS on role-ganesha in Book “Deployment Guide”, Chapter 12 “Installation of NFS Ganesha”, Section 12.1.2 “Summary of Requirements” (https://bugzilla.suse.com/show_bug.cgi?id=1107625).
Added Section 16.3.3, “Supported Operations” (https://bugzilla.suse.com/show_bug.cgi?id=1107624).
Improved auto-replacement OSDs procedure in Section 1.7, “Replacing an OSD Disk” (https://bugzilla.suse.com/show_bug.cgi?id=1107090).
Fixed libvirt keyring creation in Section 18.1, “Configuring Ceph” (https://bugzilla.suse.com/show_bug.cgi?id=1106495).
Simplified a user creation command in Section 18.1, “Configuring Ceph” https://bugzilla.suse.com/show_bug.cgi?id=1102467).
Added 'tier' to the command in Section 8.3.2, “Migrate Using Cache Tier” https://bugzilla.suse.com/show_bug.cgi?id=1102212).
Added a missing salt command in Section 1.6.2, “Removing Broken OSDs Forcefully” https://bugzilla.suse.com/show_bug.cgi?id=1100701).
Added the state.apply part to the salt in Section 1.8, “Recovering a Reinstalled OSD Node” https://bugzilla.suse.com/show_bug.cgi?id=1095937).
Added Section 17.1.1, “Enabling Secure Access to openATTIC using SSL” https://bugzilla.suse.com/show_bug.cgi?id=1083216).
Added Section 13.12, “Load Balancing the Object Gateway Servers with HAProxy” https://bugzilla.suse.com/show_bug.cgi?id=1093513).
Added Section 7.5, “Scrubbing” https://bugzilla.suse.com/show_bug.cgi?id=1079256).
Enhanced the port list for Firewall setting in Section 20.10, “Firewall Settings for Ceph” https://bugzilla.suse.com/show_bug.cgi?id=1070087).
Differed roles restarting as per DeepSea version in Section 3.2.2, “Restarting Specific Services” https://bugzilla.suse.com/show_bug.cgi?id=1091075).
Added Section 1.4, “Redeploying Monitor Nodes” https://bugzilla.suse.com/show_bug.cgi?id=1038731).
Added Section 13.9.1.1, “Dynamic Resharding” https://bugzilla.suse.com/show_bug.cgi?id=1076001).
Added Section 13.9, “Bucket Index Sharding” https://bugzilla.suse.com/show_bug.cgi?id=1076000).
Updated the Object Gateway SSL the DeepSea way in Section 13.6, “Enabling HTTPS/SSL for Object Gateways” (https://bugzilla.suse.com/show_bug.cgi?id=1083756 and https://bugzilla.suse.com/show_bug.cgi?id=1077809).
Changes in DeepSea require a modified deployment of NFS Ganesha with Object Gateway. See Section 16.3.1, “Different Object Gateway Users for NFS Ganesha” (https://bugzilla.suse.com/show_bug.cgi?id=1058821). ceph osd pool create fails if placement group limit per OSD is exceeded. See Section 8.2.2, “Create a Pool”. (https://bugzilla.suse.com/show_bug.cgi?id=1076509)
Added Section 1.8, “Recovering a Reinstalled OSD Node” (https://bugzilla.suse.com/show_bug.cgi?id=1057764).
Added a reliability warning in Section 12.1, “Runtime Configuration” (https://bugzilla.suse.com/show_bug.cgi?id=989349).
Added Section 13.2, “Deploying the Object Gateway” (https://bugzilla.suse.com/show_bug.cgi?id=1088895).
Removed lz4 from the list of compression algorithms in Section 8.5.3, “Global Compression Options” (https://bugzilla.suse.com/show_bug.cgi?id=1088450).
Added a tip on removing multiple OSDs in Section 1.6, “Removing an OSD” (https://bugzilla.suse.com/show_bug.cgi?id=1070791).
Added Section 20.3, “Stopping OSDs without Rebalancing” (https://bugzilla.suse.com/show_bug.cgi?id=1051039).
Added MDS cache size configurables in Book “Deployment Guide”, Chapter 11 “Installation of CephFS”, Section 11.2.2 “Configuring a Metadata Server” (https://bugzilla.suse.com/show_bug.cgi?id=1062692).
Added Section 13.10, “Integrating OpenStack Keystone” (https://bugzilla.suse.com/show_bug.cgi?id=1077941).
Added a tip about synchronizing iSCSI Gateway configuration in Book “Deployment Guide”, Chapter 10 “Installation of iSCSI Gateway”, Section 10.4.3 “Export RBD Images via iSCSI” (https://bugzilla.suse.com/show_bug.cgi?id=1073327).
Changes in DeepSea require a modified deployment of NFS Ganesha with Object Gateway. See Section 16.3.1, “Different Object Gateway Users for NFS Ganesha” (https://bugzilla.suse.com/show_bug.cgi?id=1058821).

D.3 November 2017 (Documentation Maintenance Update) #

General Updates #

Added Section 20.12, “Replacing Storage Disk” (Fate #321032).

Bugfixes #

Added Section 10.4, “Erasure Coded Pools with RADOS Block Device” (https://bugzilla.suse.com/show_bug.cgi?id=1075158).
Added a section about adding a disk to OSD nodes. See Section 1.5, “Adding an OSD Disk to a Node”. (https://bugzilla.suse.com/show_bug.cgi?id=1066005)
salt-run remove.osd requires OSD_ID digit without leading osd.. See Section 1.6, “Removing an OSD”.
ceph tell requires OSD_ID digit and leading osd.. See Section 22.8, “Finding Slow OSDs”.
Added Section 22.11, “/var Running Out of Space” (https://bugzilla.suse.com/show_bug.cgi?id=1069255).
Added Section 13.1, “Object Gateway Restrictions and Naming Limitations” (https://bugzilla.suse.com/show_bug.cgi?id=1067613).
Fixed rbd-mirror starting and stopping commands in Section 9.4.1, “rbd-mirror Daemon” (https://bugzilla.suse.com/show_bug.cgi?id=1068061).

D.4 October, 2017 (Release of SUSE Enterprise Storage 5.5) #

General Updates #

Removed Calamari in favor of openATTIC.
Added Section 17.4.6, “Managing NFS Ganesha” (Fate #321620).
Added Section 9.2.1, “rbdmap: Map RBD Devices at Boot Time”
Added Chapter 9, RADOS Block Device (Fate #321061).
Added Section 17.4.9, “Managing Object Gateway Users and Buckets” (Fate #320318).
Added Section 13.8, “LDAP Authentication” (Fate #321631).
Added Section 15.4, “Multiple Active MDS Daemons (Active-Active MDS)” (Fate #322976).
Added Section 17.4.7, “Managing iSCSI Gateways” (Fate #321370).
Added iSCSI Gateway and Object Gateway configuration for openATTIC, see Section 17.1.5, “Object Gateway Management” and Section 17.1.6, “iSCSI Gateway Management” (Fate #320318 and #321370).
Updated Chapter 16, NFS Ganesha: Export Ceph Data via NFS (https://bugzilla.suse.com/show_bug.cgi?id=1036495, https://bugzilla.suse.com/show_bug.cgi?id=1031444).
RBD images can now be stored in EC pools, see Section 9.1.2, “Creating a Block Device Image in an Erasure Coded Pool” (https://bugzilla.suse.com/show_bug.cgi?id=1040752).
Added section about backing up DeepSea configuration, see Book “Deployment Guide”, Chapter 6 “Backing Up the Cluster Configuration” (https://bugzilla.suse.com/show_bug.cgi?id=1046497).
Object Gateway failover and disaster recovery, see Section 13.11.12, “Failover and Disaster Recovery” (https://bugzilla.suse.com/show_bug.cgi?id=1036084).
BlueStore enables data compression for pools, see Section 8.5, “Data Compression” (FATE#318582).
CIFS export of CephFS is possible, see Book “Deployment Guide”, Chapter 13 “Exporting Ceph Data via Samba” (FATE#321622).
Added procedure for cluster reboot, see Section 1.11, “Halting or Rebooting Cluster” (https://bugzilla.suse.com/show_bug.cgi?id=1047638).
DeepSea stage 0 can update without rebooting, see Section 1.10, “Updating the Cluster Nodes”.
ceph fs replaced, see Section 15.4.3, “Decreasing the Number of Ranks” and Section 15.4.2, “Increasing the MDS Active Cluster Size” (https://bugzilla.suse.com/show_bug.cgi?id=1047638).
Added section Section 20.11, “Testing Network Performance” (FATE#321031).

Bugfixes #

The swift client package is now part of the 'Public Cloud' module in Section 13.5.1, “Accessing Object Gateway” (https://bugzilla.suse.com/show_bug.cgi?id=1057591).
Added Section 12.1, “Runtime Configuration” (https://bugzilla.suse.com/show_bug.cgi?id=1061435).
CivetWeb binds to multiple ports in Tip: Binding to Multiple Ports (https://bugzilla.suse.com/show_bug.cgi?id=1055181).
Included 3 Object Gateway options affecting performance in Section 13.4, “Configuration Parameters” (https://bugzilla.suse.com/show_bug.cgi?id=1052983).
Imported Section 1.12, “Adjusting ceph.conf with Custom Settings” and added the need for Stage 3 (https://bugzilla.suse.com/show_bug.cgi?id=1057273).
Added libvirt keyring creation in Section 18.1, “Configuring Ceph” (https://bugzilla.suse.com/show_bug.cgi?id=1055610).
Added Example 1.1, “Removing a Salt minion from the Cluster” (https://bugzilla.suse.com/show_bug.cgi?id=1054516).
Updated Section 4.3, “Watching a Cluster” (https://bugzilla.suse.com/show_bug.cgi?id=1053638).
Made Salt REST API variables optional in Section 17.1, “openATTIC Deployment and Configuration” (https://bugzilla.suse.com/show_bug.cgi?id=1054748 and https://bugzilla.suse.com/show_bug.cgi?id=1054749).
Removed oaconfig install in Section 17.1.3, “openATTIC Initial Setup” (https://bugzilla.suse.com/show_bug.cgi?id=1054747).
Added a section about displaying a pool metadata in Section 8.1, “Associate Pools with an Application” (https://bugzilla.suse.com/show_bug.cgi?id=1053327).
A list of health codes imported in Section 4.2, “Checking Cluster Health” (https://bugzilla.suse.com/show_bug.cgi?id=1052939).
Updated screenshots and related text in Section 17.4.9.1, “Adding a New Object Gateway User” and Section 17.4.9.3, “Editing Object Gateway Users” (https://bugzilla.suse.com/show_bug.cgi?id=1051814 and (https://bugzilla.suse.com/show_bug.cgi?id=1051816).
Added Object Gateway buckets in Section 17.4.9, “Managing Object Gateway Users and Buckets” (https://bugzilla.suse.com/show_bug.cgi?id=1051800).
Included cephx in mount examples in Section 15.1.3, “Mount CephFS” (https://bugzilla.suse.com/show_bug.cgi?id=1053022).
Updated and improved the description of pool deletion in Section 8.2.4, “Delete a Pool” (https://bugzilla.suse.com/show_bug.cgi?id=1052981).
Added compression algorithms description in Section 8.5.2, “Pool Compression Options” (https://bugzilla.suse.com/show_bug.cgi?id=1051457).
Replaced network diagnostics and benchmark in Section 22.10, “Poor Cluster Performance Caused by Network Problems” (https://bugzilla.suse.com/show_bug.cgi?id=1050190).
Extended Section 22.5, “'nn pg stuck inactive' Status Message” (https://bugzilla.suse.com/show_bug.cgi?id=1050183).
Mentioned pool re-creation in Section 22.4, “'Too Many PGs per OSD' Status Message” (https://bugzilla.suse.com/show_bug.cgi?id=1050178).
Fixed RGW section name in ceph.conf in Book “Deployment Guide”, Chapter 9 “Ceph Object Gateway”, Section 9.1 “Object Gateway Manual Installation” (https://bugzilla.suse.com/show_bug.cgi?id=1050170).
Updated commands output in Section 4.4, “Checking a Cluster's Usage Stats” and Section 4.3, “Watching a Cluster” (https://bugzilla.suse.com/show_bug.cgi?id=1050175).
Removed a preventive HEALTCH_WARN section in Section 4.2, “Checking Cluster Health” (https://bugzilla.suse.com/show_bug.cgi?id=1050174).
Fixed sudo in Section 13.5.2.1, “Adding S3 and Swift Users” (https://bugzilla.suse.com/show_bug.cgi?id=1050177).
Removed a reference to a RADOS striper in Section 22.2, “Sending Large Objects with rados Fails with Full OSD” (https://bugzilla.suse.com/show_bug.cgi?id=1050171).
Improved section about OSD failure because of journal failure in Section 21.5, “What Happens When a Journal Disk Fails?” (https://bugzilla.suse.com/show_bug.cgi?id=1050169).
Added a tip on zypper patch during Stage 0 in Section 1.10, “Updating the Cluster Nodes” (https://bugzilla.suse.com/show_bug.cgi?id=1050165).
Added Section 8.1, “Associate Pools with an Application” (https://bugzilla.suse.com/show_bug.cgi?id=1049940).
Improved time synchronization information in Section 22.9, “Fixing Clock Skew Warnings” (https://bugzilla.suse.com/show_bug.cgi?id=1050186).
Replaced 'erasure pool' with the correct 'erasure coded pool' (https://bugzilla.suse.com/show_bug.cgi?id=1050093).
Replaced rcceph with systemctl (https://bugzilla.suse.com/show_bug.cgi?id=1050111).
Updated CephFS mounting preparation in Section 15.1.1, “Client Preparation” (https://bugzilla.suse.com/show_bug.cgi?id=1049451).
Fixed qemu-img command in Section 18.1, “Configuring Ceph” (https://bugzilla.suse.com/show_bug.cgi?id=1047190).
Specified which DeepSea stages to run when removing roles in Section 1.3, “Removing and Reinstalling Cluster Nodes” (https://bugzilla.suse.com/show_bug.cgi?id=1047430).
Added a new DeepSea role Ceph Manager (https://bugzilla.suse.com/show_bug.cgi?id=1047472).
Adjusted intro for 12 SP3 in Section 15.1.1, “Client Preparation” (https://bugzilla.suse.com/show_bug.cgi?id=1043739).
Fixed typo in XML entity in Section 18.4, “Configuring the VM” (https://bugzilla.suse.com/show_bug.cgi?id=1042917).
Added information to re-run DeepSea Stages 2-5 for a role removal in Section 1.3, “Removing and Reinstalling Cluster Nodes” (https://bugzilla.suse.com/show_bug.cgi?id=1041899).
Added Object Gateway, iSCSI Gateway, and NFS Ganesha ports numbers that need to be open in SUSE Firewall in Section 20.10, “Firewall Settings for Ceph” (https://bugzilla.suse.com/show_bug.cgi?id=1034081).
Added description of CRUSH map tree iteration, see Section 7.3.1, “Iterating Through the Node Tree”.
Added indep parameter to CRUSH rule, see Section 7.3.2, “firstn and indep”. (https://bugzilla.suse.com/show_bug.cgi?id=1025189)
Mounting CephFS via /etc/fstab requires _netdev parameter. See Section 15.3, “CephFS in /etc/fstab” (https://bugzilla.suse.com/show_bug.cgi?id=989349)
Added tip on an existing rbdmap systemd service file in Section 9.2, “Mounting and Unmounting” (https://bugzilla.suse.com/show_bug.cgi?id=1015748).
Added an explanation to use_gmt_hitset option in Section 11.7.1.1, “Use GMT for Hit Set” (https://bugzilla.suse.com/show_bug.cgi?id=1024522).
Moved mounting CephFS back into the admin guide and added a client preparation section in Section 15.1.1, “Client Preparation” (https://bugzilla.suse.com/show_bug.cgi?id=1025447).