使用云效流水线功能实现 Go 私有库和 Proto 中心仓库自动发布

发表于 分类于

Proto & Buf

  • buf.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# yaml-language-server: $schema=https://json.schemastore.org/buf.json

version: v2
deps:
  - buf.build/googleapis/googleapis
  - buf.build/envoyproxy/protoc-gen-validate
  - buf.build/grpc-ecosystem/grpc-gateway
lint:
  use:
    - BASIC
  ignore_only:
    IMPORT_NO_PUBLIC:
      - common/common.proto
breaking:
  use:
    - FILE
  • buf.gen.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
# yaml-language-server: $schema=https://json.schemastore.org/buf.gen.json

version: v2
plugins:
  # https://github.com/protocolbuffers/protobuf-go
  - local: protoc-gen-go
    out: gen_go
    opt:
      - paths=source_relative

  # https://github.com/grpc/grpc-go
  - local: protoc-gen-go-grpc
    out: gen_go
    opt:
      - paths=source_relative

  # github.com/envoyproxy/protoc-gen-validate
  - local: protoc-gen-validate
    out: gen_go
    opt:
      - paths=source_relative
      - lang=go

  # github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway
  # - local: protoc-gen-grpc-gateway
  #   out: gen_go
  #   opt:
  #     - paths=source_relative

  # github.com/go-kratos/kratos/cmd/protoc-gen-go-http/v2
  - local: protoc-gen-go-http
    out: gen_go
    opt:
      - paths=source_relative

  # github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2
  - local: protoc-gen-openapiv2
    strategy: all
    out: gen_doc
    opt:
      - allow_merge=true
      - include_package_in_tags=true
      - fqn_for_openapi_name=true
      - use_go_templates=true
      - disable_default_errors=true
      - enums_as_ints=true
      - output_format=yaml
      - preserve_rpc_order=true

流水线

服务连接管理

webui

构建集群

webui

生成并发布 Proto

将 api-pb 所生成的 go 代码推送至 api-go 仓库，以便 goprivate 使用。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
defaultWorkspace: api_pb

sources:
  api_pb:
    type: codeup
    endpoint: git@codeup.aliyun.com:starudream/os/api-pb.git
    certificate:
      type: serviceConnection
      serviceConnection: ${云效Codeup服务连接ID}
    triggerEvents:
      - push
  api_go:
    type: codeup
    endpoint: git@codeup.aliyun.com:starudream/os/api-go.git
    certificate:
      type: serviceConnection
      serviceConnection: ${云效Codeup服务连接ID}
    triggerEvents:
      - tagPush # 防止 push 时触发该工作流

stages:
  proto:
    jobs:
      gen:
        timeoutMinutes: 30
        runsOn:
          group: private/${私有构建集群ID}
        steps:
          gen:
            step: CustomEnvironmentBuild
            with:
              image: registry.cn-shanghai.aliyuncs.com/starudream/go-dev
              privateRegistry: true
              certificate:
                type: serviceConnection
                serviceConnection: ${ACR服务连接ID}
              run: |
                go version
                buf --version
                make clean gen
                tar -czf pb_gen_${PIPELINE_ID}_${BUILD_NUMBER}.tar.gz gen_go gen_doc
                mkdir -p /cache/proto/ && mv -f pb_gen_${PIPELINE_ID}_${BUILD_NUMBER}.tar.gz /cache/proto/

      pub:
        timeoutMinutes: 30
        runsOn:
          group: private/${私有构建集群ID}
          vm: true
        needs: gen
        steps:
          go:
            step: Command
            with:
              run: |
                pushd /data/yunxiao/cache/proto/ >/dev/null
                rm -rf ${PIPELINE_ID}_${BUILD_NUMBER} && mkdir -p ${PIPELINE_ID}_${BUILD_NUMBER}
                tar -xzf pb_gen_${PIPELINE_ID}_${BUILD_NUMBER}.tar.gz -C ${PIPELINE_ID}_${BUILD_NUMBER}
                mv -f ${WORK_SPACE}/api_go/.git ${PIPELINE_ID}_${BUILD_NUMBER}/gen_go/.git

                pushd ${PIPELINE_ID}_${BUILD_NUMBER}/gen_go >/dev/null
                git config --global user.name "ci-bot"
                git config --global user.email "ci-bot@mail.starudream.cn"
                git add . && git commit -m "${CI_COMMIT_ID_1} -> ${CI_COMMIT_TITLE_1}" && git push
                popd >/dev/null

                rm -rf ${PIPELINE_ID}_${BUILD_NUMBER}
webui

GoPrivate

Configuration

1
2
3
4
go env -w CGO_ENABLED=0
go env -w GO111MODULE=on
go env -w GOPRIVATE=codeup.aliyun.com
go env -w GOPROXY=https://goproxy.cn,direct

ssh key

1
replace codeup.aliyun.com/starudream/os/api-go => codeup.aliyun.com/starudream/os/api-go.git

需要手动指定版本，不能更新，比较麻烦。

netrc

  • windows: $HOME/_netrc
  • linux: $HOME/.netrc

可以使用 go get -u 自动更新。

Backend

1
2
3
4
5
6
7
8
module codeup.aliyun.com/starudream/os/backend

go 1.23.0

require (
	codeup.aliyun.com/starudream/os/api-go v0.0.0-20241111023212-d66b05b24117
	codeup.aliyun.com/starudream/os/lib-go v0.0.0-20241111082247-43cf457f0683
)