Find the RADOS pool name and namespace for NFS Ganesha exports. The following command outputs a string of the POOL_NAME/NAMESPACE form.

cephadm@adm > ceph dashboard get-ganesha-clusters-rados-pool-namespace
cephfs_data/ganesha

By using the obtained pool name and namespace, list the RADOS objects available on that pool:

cephadm@adm > rados -p cephfs_data -N ganesha ls
conf-osd-node1
export-1
conf-osd-node2

Tip

To see how each node is configured, view its content using the following command:

cephadm@adm > cat conf-osd-node1
%url "rados://cephfs_data/ganesha/export-1"
cat conf-osd-node2
%url "rados://cephfs_data/ganesha/export-1"

In this case, both nodes will use rados://cephfs_data/ganesha/export-1. If there are multiple configurations, each node can use a different configuration.

Each export configuration is stored in a single object with the name export-ID. Use the following command to obtain the contents of the object and save it to /tmp/export-1:

cephadm@adm > rados -p cephfs_data -N ganesha get export-1 /tmp/export-1
cephadm@adm > cat /tmp/export-1
EXPORT {
    export_id = 1;
    path = "/";
    pseudo = "/cephfs";
    access_type = "RW";
    squash = "no_root_squash";
    protocols = 3, 4;
    transports = "UDP", "TCP";
    FSAL {
        name = "CEPH";
        user_id = "admin";
        filesystem = "cephfs";
        secret_access_key = "SECRET_KEY";
    }

    CLIENT {
        clients = 192.168.3.105;
        access_type = "RW";
        squash = "no_root_squash";
    }
}

Open the file /srv/pillar/ceph/stack/global.yml with the editor of your choice. Create the file if it does not exist.

The file needs to contain the following lines:

rgw_configurations:
  - rgw
  - silver
  - gold
ganesha_configurations:
  - silver
  - gold

These roles can later be assigned in the policy.cfg.

Create a file /srv/salt/ceph/rgw/users/users.d/gold.yml and add the following content:

- { uid: "gold1", name: "gold1", email: "gold1@demo.nil" }

Create a file /srv/salt/ceph/rgw/users/users.d/silver.yml and add the following content:

- { uid: "silver1", name: "silver1", email: "silver1@demo.nil" }

Now, templates for the ganesha.conf need to be created for each role. The original template of DeepSea is a good start. Create two copies:

root@master # cd /srv/salt/ceph/ganesha/files/
root@master # cp ganesha.conf.j2 silver.conf.j2
root@master # cp ganesha.conf.j2 gold.conf.j2

The new roles require keyrings to access the cluster. To provide access, copy the ganesha.j2:

root@master # cp ganesha.j2 silver.j2
root@master # cp ganesha.j2 gold.j2

Copy the keyring for the Object Gateway:

root@master # cd /srv/salt/ceph/rgw/files/
root@master # cp rgw.j2 silver.j2
root@master # cp rgw.j2 gold.j2

Object Gateway also needs the configuration for the different roles:

root@master # cd /srv/salt/ceph/configuration/files/
root@master # cp ceph.conf.rgw silver.conf
root@master # cp ceph.conf.rgw gold.conf

Assign the newly created roles to cluster nodes in the /srv/pillar/ceph/proposals/policy.cfg:

role-silver/cluster/NODE1.sls
role-gold/cluster/NODE2.sls

Replace NODE1 and NODE2 with the names of the nodes to which you want to assign the roles.

Execute DeepSea Stages 0 to 4.

Open the file /srv/pillar/ceph/rgw.sls with the editor of your choice. Create the file if it does not exist.

The file needs to contain the following lines:

rgw_configurations:
  ganesha_cfs:
    users:
      - { uid: "demo", name: "Demo", email: "demo@demo.nil" }
  ganesha_rgw:
    users:
      - { uid: "demo", name: "Demo", email: "demo@demo.nil" }

ganesha_configurations:
  - ganesha_cfs
  - ganesha_rgw

These roles can later be assigned in the policy.cfg.

Now, templates for the ganesha.conf need to be created for each role. The original template of DeepSea is a good start. Create two copies:

root@master # cd /srv/salt/ceph/ganesha/files/
root@master # cp ganesha.conf.j2 ganesha_rgw.conf.j2
root@master # cp ganesha.conf.j2 ganesha_cfs.conf.j2

Edit the ganesha_rgw.conf.j2 and remove the section:

{% if salt.saltutil.runner('select.minions', cluster='ceph', roles='mds') != [] %}
        [...]
{% endif %}

Edit the ganesha_cfs.conf.j2 and remove the section:

{% if salt.saltutil.runner('select.minions', cluster='ceph', roles=role) != [] %}
        [...]
{% endif %}

The new roles require keyrings to access the cluster. To provide access, copy the ganesha.j2:

root@master # cp ganesha.j2 ganesha_rgw.j2
root@master # cp ganesha.j2 ganesha_cfs.j2

The line caps mds = "allow *" can be removed from the ganesha_rgw.j2.

Copy the keyring for the Object Gateway:

root@master # cp /srv/salt/ceph/rgw/files/rgw.j2 \
/srv/salt/ceph/rgw/files/ganesha_rgw.j2

Object Gateway needs the configuration for the new role:

root@master # cp /srv/salt/ceph/configuration/files/ceph.conf.rgw \
/srv/salt/ceph/configuration/files/ceph.conf.ganesha_rgw

Assign the newly created roles to cluster nodes in the /srv/pillar/ceph/proposals/policy.cfg:

role-ganesha_rgw/cluster/NODE1.sls
role-ganesha_cfs/cluster/NODE1.sls

Replace NODE1 and NODE2 with the names of the nodes to which you want to assign the roles.

Execute DeepSea Stages 0 to 4.