5. Swift文件存储对接Ceph RadosGW
实际上应该是
Ceph RadosGW
对接OpenStack Keystone
身份认证,因为RadosGW
提供了和Swift
兼容的API,所有二者api是一样的,直接做个负载均衡到RadosGW
就成了一个swift接口
# 1. 前言
ceph.conf
配置中的键如果是空格形式的(形如“rgw keystone api version”),这种是 ceph octopus
版本最后一次出现,在 pacific
版本改为下划线形式。
重点:
如果使用 kolla-ansible
部署 RadosGW
兼容的 Swift
,不要去找那种 “手动部署RadosGW对接Swift” 的教程,因为kolla-ansible
已经为你准备好了 OpenStack
这边的一切配置。所有有关 OpenStack
这边的操作都 不要做!!!
RadosGW 兼容的 Swift:实际上没有部署 Swift,可以理解为把原本 Swift 的 api 重定向到了 RadosGW。而 RadosGW 有着和 Swift一样的api
# 2. Ceph配置
RadosGW安装配置这里不再赘述,请参阅:
# 2.1 检查Ceph.conf
做完之后你的ceph.conf应该大致如下:
建议取消所有缩进
[global]
auth_client_required = cephx
auth_cluster_required = cephx
auth_service_required = cephx
cluster_network = 10.10.11.1/24
fsid = 59063611-9b12-4807-bf9d-ecfa60480d94
mon_allow_pool_delete = true
mon_host = 10.10.1.51
ms_bind_ipv4 = true
ms_bind_ipv6 = false
osd_pool_default_min_size = 2
osd_pool_default_size = 2
public_network = 10.10.1.51/24
[client]
keyring = /etc/ceph/priv/$cluster.$name.keyring
[client.radosgw.pve1]
host = pve1
keyring = /etc/ceph/priv/ceph.client.radosgw.keyring
# common
rgw_keystone_url = 10.10.1.250:5000
rgw_keystone_api_version = 3
# rgw_keystone_verify_ssl = false
# v3
rgw_keystone_admin_domain = Default
rgw_keystone_admin_project = admin
rgw_keystone_admin_token = gAAAAABj4bqH7oqC7X45z41YGs
# other
# rgw_s3_auth_use_keystone = true
# rgw_swift_account_in_url = true
[mds]
keyring = /var/lib/ceph/mds/ceph-$id/keyring
[mds.pve1]
host = pve1
mds_standby_for_name = ceph
[mon.pve1]
public_addr = 10.10.1.51
# 2.2 对接Keystone
这里需要先安装一个cli client工具。进入部署机,激活python虚拟环境,source admin授权文件,安装
. /path/to/venv/bin/activate
. /etc/kolla/admin-openrc.sh
pip install python-swiftclient
验证安装
swift --help
# 2.3 获取Auth Token
swift auth
export OS_STORAGE_URL=http://10.10.1.250:6780/swift/v1
export OS_AUTH_TOKEN=gAAAAABj4cEubXFK1PLGUicIMKMft6TiWitNcGhzHJxS2CWbetUEYIHbCJMx3v_iLQ142ZgjQ9KvtwLwoDrP-_RctuP45O9FTVr1dAVWrEKZngXpxX2uRVSELz7FP27vmZqtN0
感兴趣的可以再试试下面两个命令:
swift auth -v
swift auth --debug
把 OS_AUTH_TOKEN
记录下来,回到 Ceph 部署主机
echo "gAAAAABj4cID5BqJhJcRPfAJniQ-eqt66praFVJ2ZwSGUMd6QgdUplNUJlih-AGOzYw5t5HOWHiCsTGsMfU2uJpBwbXsC_2Y8hpqSwqqICth8xfLtnivPcKaT__K9CNASb92REHMECmQI" >> /etc/ceph/keystone_admin_token
# 2.4 配置RadosGW
在Ceph 部署主机节点中,打开ceph.conf,在[client.radosgw.pve1]节点下新增:
# common
rgw_keystone_url = 10.10.1.250:5000
rgw_keystone_api_version = 3
# rgw_keystone_verify_ssl = false
# v3
rgw_keystone_admin_domain = Default
rgw_keystone_admin_project = admin
rgw_keystone_admin_token = gAAAAABj4bqH7oqC7X45z41YGsrBR5U5RfMbHylk6nlk2ZquW58h0
# other
# rgw_s3_auth_use_keystone = true
# rgw_swift_account_in_url = true
完成后,整个ceph.conf看起来如下
[global]
auth_client_required = cephx
auth_cluster_required = cephx
auth_service_required = cephx
cluster_network = 10.10.11.1/24
fsid = 59063611-9b12-4807-bf9d-ecfa60480d94
mon_allow_pool_delete = true
mon_host = 10.10.1.51
ms_bind_ipv4 = true
ms_bind_ipv6 = false
osd_pool_default_min_size = 2
osd_pool_default_size = 2
public_network = 10.10.1.51/24
[client]
keyring = /etc/pve/priv/$cluster.$name.keyring
[client.radosgw.pve1]
host = pve1
keyring = /etc/pve/priv/ceph.client.radosgw.keyring
# common
rgw_keystone_url = 10.10.1.250:5000
rgw_keystone_api_version = 3
# rgw_keystone_verify_ssl = false
# v3
rgw_keystone_admin_domain = Default
rgw_keystone_admin_project = admin
rgw_keystone_admin_token = gAAAAABj4bqH7oqC7X45z41YGs
# other
# rgw_s3_auth_use_keystone = true
# rgw_swift_account_in_url = true
[mds]
keyring = /var/lib/ceph/mds/ceph-$id/keyring
[mds.pve1]
host = pve1
mds standby for name = pve
[mon.pve1]
public_addr = 10.10.1.51
上传保存,重启 RadosGW (容器方式)
# 3. kolla配置
# 3.1 启用 Ceph RadosGW 集成
在部署机,/etc/kolla/global.yml
enable_ceph_rgw: true
# 3.2 添加rgw主机
在 enable_ceph_rgw: "yes" 下面一行,增加
ceph_rgw_hosts:
- host: rgw01
ip: 10.10.1.51
port: 7480
host 是名字随意,不需要解析dns,但是一定要填写ip
端口7480,是 RadosGW
默认端口,可以在 RadosGW
文档中查询到 rgw_frontends
属性
# 4. 重新部署
kolla-ansible -i ./multinode deploy
# 5. 验证
# 5.1 验证负载均衡
haproxy报告查看:
radosgw_back
都是绿色就行
# 5.2 验证客户端
回到部署机,输入 swift stat
(venv) root@ttt:~# swift stat
Account: v1
Containers: 0
Objects: 0
Bytes: 0
Containers in policy "default-placement": 0
Objects in policy "default-placement": 0
Bytes in policy "default-placement": 0
Objects in policy "default-placement-bytes": 0
Bytes in policy "default-placement-bytes": 0
X-Timestamp: 1675740884.45155
X-Account-Bytes-Used-Actual: 0
X-Trans-Id: tx000008f5efad48bd19d87-0063e1c6d4-96a42-default
X-Openstack-Request-Id: tx000008f5efad48bd19d87-0063e1c6d4-96a42-default
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
(venv) root@ttt:~#
# 5.3 验证 Horizon
重新登录Horizon,项目>对象存储>容器
随便创建一个
创建成功
上传文件,再下载,没问题