harbor/registry设置Ceph对象存储

harbor/registry设置Ceph对象存储

纯干货分享
http://jiangjiang.space 回忆书签,转载请注明出处,谢谢

1.创建ceph账户和swift子账户

安装ceph(略)
创建ceph账户

radosgw-admin subuser create --uid demo --subuser=demo:swift --access=full --secret=secretkey --key-type=swift

创建swift api 子账户

$ radosgw-admin key create --subuser=demo:swift --key-type=swift --gen-secret
{
    "user_id": "demo",
    "display_name": "ceph rgw demo user",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [
        {
            "id": "demo:swift",
            "permissions": "full-control"
        }
    ],
    "keys": [
        {
            "user": "demo",
            "access_key": "5Y1B1SIJ2YHKEHO5U36B",
            "secret_key": "nrIvtPqUj7pUlccLYPuR3ntVzIa50DToIpe7xFjT"
        }
    ],
    "swift_keys": [
        {
            "user": "demo:swift",
            "secret_key": "aCgVTx3Gfz1dBiFS4NfjIRmvT0sgpHDP6aa0Yfrh"
        }
    ],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    },
        "temp_url_keys": []
}

创建存储筒:
用S3客户端创建一个存储筒,将来push上来的镜像就存在这里。


纯干货分享
http://jiangjiang.space 回忆书签,转载请注明出处,谢谢

2. harbor中的registry

Harbor中包含一套标准的docker registry, 修改registry设置只需要修改这个registry的配置文件和compose文件即可。
按照harbor的compose文件

- harbor
  registry:
    image: vmware/registry:2.6.2-photon
    container_name: registry
    restart: always
    volumes:
      - /data:/storage:z
      - ./common/config/registry/:/etc/registry/:z   ## 这里看到registry的配置文件

修改./common/config/registry/config.yml文件:

 #filesystem:
    #    rootdirectory: /storage
    swift:
         username: demo:swift   # 刚才创建的swift子账户
         password: xxxxxxxxtzhNH # 刚才生成出的key
         authurl: http://ceph01/auth/v1.0  # swift api url
         tenant: demo  # 这个一定要写,否则报错
         container: regswfit # 注意这里的container并非dockercontainer

注释 filesystem,由于registry直接用S3方式连接有问题,因此用swift api接口对接。
增加swift的配置, 注意这些键值一个都不能少, 后面会有少写key的报错信息。
我是一点一点磕碰出来的配置。

注意,这里的container不是docker container, 其实就是S3的存储筒, 这个筒要事先创建好,否则也报错。

3. 重启registry容器

# 找到 registry 容器
docker  ps

1ea3ee9c0f8e        vmware/registry:2.6.2-photon              "/entrypoint.sh se..."   3 hours ago         Up 2 hours

# 停止容器
docker  stop  1ea
# 启动容器
docker  start 1ea

观察日志看是否有报错:

tail -f /var/log/harbor/日期/registry.log
# 报错对应信息见附录

4. 测试harbor

推送镜像

docker push registry.demo.cn/pub/ubuntu:16.04
The push refers to a repository [registry.demo.cn/pub/ubuntu]
49907af65b0a: Pushed 
4589f96366e6: Pushed 
b97229212d30: Pushed 
cd181336f142: Pushed 
0f5ff0cf6a1c: Pushed 

5. 附录

报错1:

Nov  2 18:02:54 172.29.248.1 registry[1095]: panic: No container parameter provided
Nov  2 18:02:54 172.29.248.1 registry[1095]: 
Nov  2 18:02:54 172.29.248.1 registry[1095]: goroutine 1 [running]:
Nov  2 18:02:54 172.29.248.1 registry[1095]: panic(0xb7dc40, 0xc4203f9570)
Nov  2 18:02:54 172.29.248.1 registry[1095]: #011/usr/local/go/src/runtime/panic.go:500 +0x1a1
Nov  2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/registry/handlers.NewApp(0x1067820, 0xc4203851d0, 0xc4202bf180, 0x1067820)
Nov  2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/handlers/app.go:123 +0x3908
Nov  2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/registry.NewRegistry(0x7f13ea120198, 0xc4203851d0, 0xc4202bf180, 0xe, 0x0, 0x0)
Nov  2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/registry.go:86 +0x213
Nov  2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/registry.glob..func1(0x108f1a0, 0xc42036acd0, 0x1, 0x1)
Nov  2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/registry.go:55 +0x106
Nov  2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).execute(0x108f1a0, 0xc42036ac80, 0x1, 0x1, 0x108f1a0, 0xc42036ac80)
Nov  2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:495 +0x190
Nov  2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).Execute(0x108f340, 0xc4201b7f40, 0xc4200001a0)
Nov  2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:560 +0x3c3
Nov  2 18:02:54 172.29.248.1 registry[1095]: main.main()
Nov  2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/cmd/registry/main.go:24 +0x2d

原因: 没有写 container(也就是s3 得存储桶)

报错2:

Nov  2 18:09:31 172.29.248.1 registry[1095]: panic: Failed to create container registry (Bad Request)
Nov  2 18:09:31 172.29.248.1 registry[1095]: 
Nov  2 18:09:31 172.29.248.1 registry[1095]: goroutine 1 [running]:
Nov  2 18:09:31 172.29.248.1 registry[1095]: panic(0xb7dc40, 0xc420295190)
Nov  2 18:09:31 172.29.248.1 registry[1095]: #011/usr/local/go/src/runtime/panic.go:500 +0x1a1
Nov  2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/registry/handlers.NewApp(0x1067820, 0xc420397800, 0xc42037ca80, 0x1067820)
Nov  2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/handlers/app.go:123 +0x3908
Nov  2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/registry.NewRegistry(0x7f383810c128, 0xc420397800, 0xc42037ca80, 0xe, 0x0, 0x0)
Nov  2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/registry.go:86 +0x213
Nov  2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/registry.glob..func1(0x108f1a0, 0xc42030da30, 0x1, 0x1)
Nov  2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/registry.go:55 +0x106
Nov  2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).execute(0x108f1a0, 0xc42030d9e0, 0x1, 0x1, 0x108f1a0, 0xc42030d9e0)
Nov  2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:495 +0x190
Nov  2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).Execute(0x108f340, 0xc4201b9f40, 0xc4200001a0)
Nov  2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:560 +0x3c3
Nov  2 18:09:31 172.29.248.1 registry[1095]: main.main()
Nov  2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/cmd/registry/main.go:24 +0x2d

原因: 没有写tenant

报错3:

Swift authentication failed: HTTP Error: 405: 405 Method Not Allowed

原因: 存储筒不存在

纯干货分享
http://jiangjiang.space 回忆书签,转载请注明出处,谢谢