OPSaid – Argo Workflows

Docs: 快速开始

Tue, 16 May 2023 10:35:39 +0800

简要概述

工作流编排。

均基于 “argo workflows/v3.4.17” 版本分析。

安装部署

登录任一控制节点：

kubectl apply -k /etc/kubernetes/addons/argo-workflows/

验证部署详情：

kubectl get pods -n argo -w

工作流类型

Workflow

它定义了要执行的工作流程；
它存储工作流的状态。

见数据结构 Workflow，一个最简单的示例：

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
 generateName: hello-world-
spec:
 entrypoint: main
 templates:
 - name: main
 container:
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ echo ]
 args: ["hello world"]

其中用户定义的流程在 “spec.templates” 集合内，“spec.entrypoint” 表示该工作流执行的入口，类似函数的 “main” 功能。

创建语句：

argo submit hello-world.yaml

或者

kubectl create -f hello-world.yaml

WorkflowTemplate

是 “Workflow” 工作流的模版，一般用于创建常用流程，然后根据参数在生成具体的工作流执行任务。

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
 name: hello-world
 namespace: default
spec:
 entrypoint: main

 arguments:
 parameters:
 - name: name
 - name: price
 - name: from
 value: create by workflow template

 templates:
 - name: main
 inputs:
 parameters:
 - name: name
 - name: price
 - name: from
 container:
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args:
 - "echo "
 - "hello: {{ inputs.parameters.name }}"
 - "age: {{ inputs.parameters.price }}"
 - "from: {{ inputs.parameters.from }}"
 outputs:
 parameters:
 - name: name
 value: "{{ inputs.parameters.name }}"
 - name: price
 value: "{{ inputs.parameters.price }}"
 - name: from
 value: "{{ inputs.parameters.from }}"

创建语句：

kubectl apply -f hello-world.yaml

ClusterWorkflowTemplate

同 “WorkflowTemplate”，但该资源是 k8s 集群范围内可用，没有用户空间隔离。

apiVersion: argoproj.io/v1alpha1
kind: ClusterWorkflowTemplate
metadata:
 name: hello-world
spec:
 ...

CronWorkflow

按照一定的计划任务去运行工作流。

---
apiVersion: argoproj.io/v1alpha1
kind: CronWorkflow
metadata:
 name: my-test-9
spec:
 schedule: "* * * * *"
 concurrencyPolicy: "Replace"
 startingDeadlineSeconds: 0
 workflowSpec:
 entrypoint: main
 templates:
 - name: main
 container:
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [sh, -c]
 args: ["date; sleep 10"]

创建语句：

argo cron create hello-world.yaml

或者

kubectl create -f hello-world.yaml

配置参数与数据结构说明参考 CronWorkflow。

任务模版类型

容器类型

通过属性 “spec.templates[_].container” 定义：

...
spec:
 templates:
 - name: tmpl-container
 container:
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ echo ]
 args: ["hello world"]

是最常用的模板类型，它将启动一个容器用于作业执行。

脚本类型

通过属性 “spec.templates[_].script” 定义：

...
spec:
 templates:
 - name: tmpl-script 
 script:
 image: ccr.ccs.tencentyun.com/opsaid/python:alpine3.6
 command:
 - python
 source: |
 import random
 i = random.randint(1, 100)
 print(i)

启动容器并执行 “spec.templates[_].script.source” 脚本内容，最终把结果写入变量 {{tasks.<NAME>.outputs.result}} 或者 {{steps.<NAME>.outputs.result}} 以供后续流程使用。

资源类型

...
spec:
 templates:
 - name: tmpl-resource
 resource:
 action: create
 manifest: |
 apiVersion: v1
 kind: ConfigMap
 metadata:
 #generateName: owned-eg-
 name: tmpl-resource
 data:
 some: value

通过属性 “spec.templates[_].resource” 定义。

这里是指 k8s 集群内的资源。

挂起类型

通过属性 “spec.templates[_].suspend” 定义：

...
spec:
 templates:
 - name: tmpl-suspend
 suspend:
 duration: "20s"

对于流水线执行到此步骤进行等待一段时间或者人工审批确认后在执行后续步骤。

步骤类型

通过属性 “spec.templates[_].steps” 定义：

...
spec:
 templates:
 - name: tmpl-steps
 steps:
 - - name: step1
 template: tmpl-container
 - - name: step2
 template: tmpl-script
 - name: step2-1
 template: tmpl-script

把大任务分解为一系列子步骤进行执行，外部列表将顺序运行（两个中横行，既 “- - name:” 下任务），内部列表将并行运行（一个中横行，既 “- name:” 任务，属于某个 “- -” 下的子任务）。如果需要逐一运行内部任务，需使用同步功能。

也可以设置多种选项来控制执行，如 “when:” 子句有条件地执行步骤。

有向无环图

通过属性 “spec.templates[_].dag” 定义：

...
spec:
 templates:
 - name: tmpl-steps
 dag:
 tasks:
 - name: A
 template: tmpl-container
 - name: B
 dependencies: [A]
 template: tmpl-container
 - name: C
 dependencies: [A]
 template: tmpl-container
 - name: D
 dependencies: [B, C]
 template: tmpl-container

可以定义每个任务之间的依赖关系，并设置在特定任务开始之前必须完成哪些其他任务。

HTTP 类型

通过属性 “spec.templates[_].http” 定义：

...
spec:
 entrypoint: main
 templates:
 - name: main
 steps:
 - - name: get-baidu
 template: http
 arguments:
 parameters:
 - name: url
 value: www.baidu.com

 - name: http
 inputs:
 parameters:
 - name: url

 http:
 method: GET
 url: "{{ inputs.parameters.url }}"
 insecureSkipVerify: true
 successCondition: "response.statusCode == 200"

实际上也是以镜像 “quay.io/argoproj/argoexec:v3.4.7” 启动容器，然后执行 http 动作。

容器集合类型

通过属性 “spec.templates[_].containerSet” 定义：

...
spec:
 entrypoint: main
 templates:
 - name: main
 volumes:
 - name: workspace
 emptyDir: {}

 containerSet:
 volumeMounts:
 - mountPath: /workspace
 name: workspace

 containers:
 - name: task-a
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: ["echo task-a >> /workspace/hello.txt"]
 - name: task-b
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: ["echo task-b >> /workspace/hello.txt"]
 - name: task-c
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: ["echo task-c >> /workspace/hello.txt"]
 dependencies: [ task-a, task-b ]
 - name: task-d
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: ["cat /workspace/hello.txt"]
 dependencies: [ task-b ]
 - name: main
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: ["echo main >> /workspace/hello.txt"]
 dependencies: [task-c, task-d]

 outputs:
 parameters:
 - name: hello
 valueFrom:
 path: /workspace/hello.txt

与以上容器类型区别在于可在单个 Pod 里运行多个容器，这些容器均运行同一个宿主上，所以可以共享存储卷等比较灵活。同时流水线入口步骤必须命名为 “main”。

Docs: 接口定义

Tue, 16 May 2023 10:35:39 +0800

简要概述

源代码定义

官方文档说明

数据结构

Workflow

https://github.com/argoproj/argo-workflows/blob/main/pkg/apis/workflow/v1alpha1/workflow_types.go

// Workflow is the definition of a workflow resource
type Workflow struct {
 metav1.TypeMeta `json:",inline"`
 metav1.ObjectMeta `json:"metadata" protobuf:"bytes,1,opt,name=metadata"`
 Spec WorkflowSpec `json:"spec" protobuf:"bytes,2,opt,name=spec "`
 Status WorkflowStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
}

WorkflowTemplate

https://github.com/argoproj/argo-workflows/blob/main/pkg/apis/workflow/v1alpha1/workflow_template_types.go

type WorkflowTemplate struct {
 metav1.TypeMeta `json:",inline"`
 metav1.ObjectMeta `json:"metadata" protobuf:"bytes,1,opt,name=metadata"`
 Spec WorkflowSpec `json:"spec" protobuf:"bytes,2,opt,name=spec"`
}

ClusterWorkflowTemplate

https://github.com/argoproj/argo-workflows/blob/main/pkg/apis/workflow/v1alpha1/cluster_workflow_template_types.go

type ClusterWorkflowTemplate struct {
 metav1.TypeMeta `json:",inline"`
 metav1.ObjectMeta `json:"metadata" protobuf:"bytes,1,opt,name=metadata"`
 Spec WorkflowSpec `json:"spec" protobuf:"bytes,2,opt,name=spec"`
}

CronWorkflow

https://github.com/argoproj/argo-workflows/blob/main/pkg/apis/workflow/v1alpha1/cron_workflow_types.go

// CronWorkflow is the definition of a scheduled workflow resource
type CronWorkflow struct {
 metav1.TypeMeta `json:",inline"`
 metav1.ObjectMeta `json:"metadata" protobuf:"bytes,1,opt,name=metadata"`
 Spec CronWorkflowSpec `json:"spec" protobuf:"bytes,2,opt,name=spec"`
 Status CronWorkflowStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
}

// CronWorkflowSpec is the specification of a CronWorkflow
type CronWorkflowSpec struct {
 // WorkflowSpec is the spec of the workflow to be run
 WorkflowSpec WorkflowSpec `json:"workflowSpec" protobuf:"bytes,1,opt,name=workflowSpec,casttype=WorkflowSpec"`

 // Schedule is a schedule to run the Workflow in Cron format
 // 必填，同 linux cron 语法：https://en.wikipedia.org/wiki/Cron
 Schedule string `json:"schedule" protobuf:"bytes,2,opt,name=schedule"`

 // ConcurrencyPolicy is the K8s-style concurrency policy that will be used
 // 同时存在多个工作流时要执行的策略，可取：Allow Replace Forbid
 // Allow 允许新任务同时执行
 // Replace 移除所有旧任务之后在执行新工作流
 // Forbid 如果旧任务未完成则不允许继续创建
 ConcurrencyPolicy ConcurrencyPolicy `json:"concurrencyPolicy,omitempty" protobuf:"bytes,3,opt,name=concurrencyPolicy,casttype=ConcurrencyPolicy"`

 // Suspend is a flag that will stop new CronWorkflows from running if set to true
 // 设置为 'true' 则停止该工作流调度
 Suspend bool `json:"suspend,omitempty" protobuf:"varint,4,opt,name=suspend"`

 // StartingDeadlineSeconds is the K8s-style deadline that will limit the time a CronWorkflow will be run after its
 // original scheduled time if it is missed.
 // 上次成功运行至当前的最大秒数，如在此期间错过的任务将被执行，比如 "workflow-controller" 故障导致
 StartingDeadlineSeconds *int64 `json:"startingDeadlineSeconds,omitempty" protobuf:"varint,5,opt,name=startingDeadlineSeconds"`

 // SuccessfulJobsHistoryLimit is the number of successful jobs to be kept at a time
 // 对已成功的工作流保留历史数量
 SuccessfulJobsHistoryLimit *int32 `json:"successfulJobsHistoryLimit,omitempty" protobuf:"varint,6,opt,name=successfulJobsHistoryLimit"`

 // FailedJobsHistoryLimit is the number of failed jobs to be kept at a time
 // 对已失败的工作流保留历史数量
 FailedJobsHistoryLimit *int32 `json:"failedJobsHistoryLimit,omitempty" protobuf:"varint,7,opt,name=failedJobsHistoryLimit"`

 // Timezone is the timezone against which the cron schedule will be calculated, e.g. "Asia/Tokyo". Default is machine's local time.
 // 计划任务依赖的时区
 Timezone string `json:"timezone,omitempty" protobuf:"bytes,8,opt,name=timezone"`

 // WorkflowMetadata contains some metadata of the workflow to be run
 WorkflowMetadata *metav1.ObjectMeta `json:"workflowMetadata,omitempty" protobuf:"bytes,9,opt,name=workflowMeta"`
}

type ConcurrencyPolicy string

const (
 AllowConcurrent ConcurrencyPolicy = "Allow"
 ForbidConcurrent ConcurrencyPolicy = "Forbid"
 ReplaceConcurrent ConcurrencyPolicy = "Replace"
)

WorkflowSpec

// WorkflowSpec is the specification of a Workflow.
type WorkflowSpec struct {
 // Templates is a list of workflow templates used in a workflow
 // +patchStrategy=merge
 // +patchMergeKey=name
 // 一个工作流由这些子任务组成
 Templates []Template `json:"templates,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,1,opt,name=templates"`

 // Entrypoint is a template reference to the starting point of the workflow.
 // 该工作流执行的入口，类似函数的 "main" 功能
 Entrypoint string `json:"entrypoint,omitempty" protobuf:"bytes,2,opt,name=entrypoint"`

 // Arguments contain the parameters and artifacts sent to the workflow entrypoint
 // Parameters are referencable globally using the 'workflow' variable prefix.
 // e.g. {{workflow.parameters.myparam}}
 // 用于配置请求或附件参数，可在下游通过变量 "{{workflow.parameters.myparam}}" 获取
 Arguments Arguments `json:"arguments,omitempty" protobuf:"bytes,3,opt,name=arguments"`

 // ServiceAccountName is the name of the ServiceAccount to run all pods of the workflow as.
 // 流水线 POD 运行的 sa 账户
 ServiceAccountName string `json:"serviceAccountName,omitempty" protobuf:"bytes,4,opt,name=serviceAccountName"`

 // AutomountServiceAccountToken indicates whether a service account token should be automatically mounted in pods.
 // ServiceAccountName of ExecutorConfig must be specified if this value is false.
 // 是否挂载 sa 的 token 至容器内
 AutomountServiceAccountToken *bool `json:"automountServiceAccountToken,omitempty" protobuf:"varint,28,opt,name=automountServiceAccountToken"`

 // Executor holds configurations of executor containers of the workflow.
 // TODO;
 Executor *ExecutorConfig `json:"executor,omitempty" protobuf:"bytes,29,opt,name=executor"`

 // Volumes is a list of volumes that can be mounted by containers in a workflow.
 // +patchStrategy=merge
 // +patchMergeKey=name
 // 挂载至工作流任务中 POD 的卷，见[Pod Volume](/docs/cloud-native/kubernetes/resources/pods-volume/)
 Volumes []apiv1.Volume `json:"volumes,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,5,opt,name=volumes"`

 // VolumeClaimTemplates is a list of claims that containers are allowed to reference.
 // The Workflow controller will create the claims at the beginning of the workflow
 // and delete the claims upon completion of the workflow
 VolumeClaimTemplates []apiv1.PersistentVolumeClaim `json:"volumeClaimTemplates,omitempty" protobuf:"bytes,6,opt,name=volumeClaimTemplates"`

 // Parallelism limits the max total parallel pods that can execute at the same time in a workflow
 // 同一时间内最大可并行执行的任务数量
 Parallelism *int64 `json:"parallelism,omitempty" protobuf:"bytes,7,opt,name=parallelism"`

 // ArtifactRepositoryRef specifies the configMap name and key containing the artifact repository config.
 ArtifactRepositoryRef *ArtifactRepositoryRef `json:"artifactRepositoryRef,omitempty" protobuf:"bytes,8,opt,name=artifactRepositoryRef"`

 // Suspend will suspend the workflow and prevent execution of any future steps in the workflow
 // 挂起工作流
 Suspend *bool `json:"suspend,omitempty" protobuf:"bytes,9,opt,name=suspend"`

 // NodeSelector is a selector which will result in all pods of the workflow
 // to be scheduled on the selected node(s). This is able to be overridden by
 // a nodeSelector specified in the template.
 // 调度任务至关联的主机节点
 NodeSelector map[string]string `json:"nodeSelector,omitempty" protobuf:"bytes,10,opt,name=nodeSelector"`

 // Affinity sets the scheduling constraints for all pods in the workflow.
 // Can be overridden by an affinity specified in the template
 // 亲和性配置，见 [Pod Affinity](/docs/cloud-native/kubernetes/resources/pods-affinity/)
 Affinity *apiv1.Affinity `json:"affinity,omitempty" protobuf:"bytes,11,opt,name=affinity"`

 // Tolerations to apply to workflow pods.
 // +patchStrategy=merge
 // +patchMergeKey=key
 // 污点，见 [Toleration](/docs/cloud-native/kubernetes/resources/pods/#toleration)
 Tolerations []apiv1.Toleration `json:"tolerations,omitempty" patchStrategy:"merge" patchMergeKey:"key" protobuf:"bytes,12,opt,name=tolerations"`

 // ImagePullSecrets is a list of references to secrets in the same namespace to use for pulling any images
 // in pods that reference this ServiceAccount. ImagePullSecrets are distinct from Secrets because Secrets
 // can be mounted in the pod, but ImagePullSecrets are only accessed by the kubelet.
 // More info: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
 // +patchStrategy=merge
 // +patchMergeKey=name
 // 拉取镜像的密钥文件
 ImagePullSecrets []apiv1.LocalObjectReference `json:"imagePullSecrets,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,13,opt,name=imagePullSecrets"`

 // Host networking requested for this workflow pod. Default to false.
 // 工作流使用主机网络
 HostNetwork *bool `json:"hostNetwork,omitempty" protobuf:"bytes,14,opt,name=hostNetwork"`

 // Set DNS policy for workflow pods.
 // Defaults to "ClusterFirst".
 // Valid values are 'ClusterFirstWithHostNet', 'ClusterFirst', 'Default' or 'None'.
 // DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy.
 // To have DNS options set along with hostNetwork, you have to specify DNS policy
 // explicitly to 'ClusterFirstWithHostNet'.
 // 工作流 pod 使用的 dns 策略，见[PodDNSConfig](/docs/cloud-native/kubernetes/resources/pods/#poddnsconfig)
 DNSPolicy *apiv1.DNSPolicy `json:"dnsPolicy,omitempty" protobuf:"bytes,15,opt,name=dnsPolicy"`

 // PodDNSConfig defines the DNS parameters of a pod in addition to
 // those generated from DNSPolicy.
 // 工作流 pod 使用的 dns 策略，见[PodDNSConfig](/docs/cloud-native/kubernetes/resources/pods/#poddnsconfig)
 DNSConfig *apiv1.PodDNSConfig `json:"dnsConfig,omitempty" protobuf:"bytes,16,opt,name=dnsConfig"`

 // OnExit is a template reference which is invoked at the end of the
 // workflow, irrespective of the success, failure, or error of the
 // primary workflow.
 // 在工作流结束时调用，无论是否执行成功或失败、错误，一般用于消息推送
 OnExit string `json:"onExit,omitempty" protobuf:"bytes,17,opt,name=onExit"`

 // TTLStrategy limits the lifetime of a Workflow that has finished execution depending on if it
 // Succeeded or Failed. If this struct is set, once the Workflow finishes, it will be
 // deleted after the time to live expires. If this field is unset,
 // the controller config map will hold the default values.
 // 工作流生命周期
 TTLStrategy *TTLStrategy `json:"ttlStrategy,omitempty" protobuf:"bytes,30,opt,name=ttlStrategy"`

 // Optional duration in seconds relative to the workflow start time which the workflow is
 // allowed to run before the controller terminates the workflow. A value of zero is used to
 // terminate a Running workflow
 // 允许工作流最长执行时间，默认没有时间限制
 ActiveDeadlineSeconds *int64 `json:"activeDeadlineSeconds,omitempty" protobuf:"bytes,19,opt,name=activeDeadlineSeconds"`

 // Priority is used if controller is configured to process limited number of workflows in parallel. Workflows with higher priority are processed first.
 // 并行处理有限数量的工作流程中，优先级越高越先执行
 Priority *int32 `json:"priority,omitempty" protobuf:"bytes,20,opt,name=priority"`

 // Set scheduler name for all pods.
 // Will be overridden if container/script template's scheduler name is set.
 // Default scheduler will be used if neither specified.
 // +optional
 // 使用 k8s 集群调度程序名称
 SchedulerName string `json:"schedulerName,omitempty" protobuf:"bytes,21,opt,name=schedulerName"`

 // PodGC describes the strategy to use when deleting completed pods
 // 任务完成后 pod 是否删除策略
 PodGC *PodGC `json:"podGC,omitempty" protobuf:"bytes,22,opt,name=podGC"`

 // PriorityClassName to apply to workflow pods.
 // TODO;
 PodPriorityClassName string `json:"podPriorityClassName,omitempty" protobuf:"bytes,23,opt,name=podPriorityClassName"`

 // Priority to apply to workflow pods.
 // DEPRECATED: Use PodPriorityClassName instead.
 PodPriority *int32 `json:"podPriority,omitempty" protobuf:"bytes,24,opt,name=podPriority"`

 // +patchStrategy=merge
 // +patchMergeKey=ip
 // 见 [PodSpec](/docs/cloud-native/kubernetes/resources/pods/#podspec)
 HostAliases []apiv1.HostAlias `json:"hostAliases,omitempty" patchStrategy:"merge" patchMergeKey:"ip" protobuf:"bytes,25,opt,name=hostAliases"`

 // SecurityContext holds pod-level security attributes and common container settings.
 // Optional: Defaults to empty. See type description for default values of each field.
 // +optional
 // 容器安全上下文，见 [Pod Security](/docs/cloud-native/kubernetes/resources/pods-security/)
 SecurityContext *apiv1.PodSecurityContext `json:"securityContext,omitempty" protobuf:"bytes,26,opt,name=securityContext"`

 // PodSpecPatch holds strategic merge patch to apply against the pod spec. Allows parameterization of
 // container fields which are not strings (e.g. resource limits).
 PodSpecPatch string `json:"podSpecPatch,omitempty" protobuf:"bytes,27,opt,name=podSpecPatch"`

 // PodDisruptionBudget holds the number of concurrent disruptions that you allow for Workflow's Pods.
 // Controller will automatically add the selector with workflow name, if selector is empty.
 // Optional: Defaults to empty.
 // +optional
 PodDisruptionBudget *policyv1.PodDisruptionBudgetSpec `json:"podDisruptionBudget,omitempty" protobuf:"bytes,31,opt,name=podDisruptionBudget"`

 // Metrics are a list of metrics emitted from this Workflow
 // 该工作流关联的指标
 Metrics *Metrics `json:"metrics,omitempty" protobuf:"bytes,32,opt,name=metrics"`

 // Shutdown will shutdown the workflow according to its ShutdownStrategy
 // TODO；关闭的策略
 Shutdown ShutdownStrategy `json:"shutdown,omitempty" protobuf:"bytes,33,opt,name=shutdown,casttype=ShutdownStrategy"`

 // WorkflowTemplateRef holds a reference to a WorkflowTemplate for execution
 // TODO;
 WorkflowTemplateRef *WorkflowTemplateRef `json:"workflowTemplateRef,omitempty" protobuf:"bytes,34,opt,name=workflowTemplateRef"`

 // Synchronization holds synchronization lock configuration for this Workflow
 // 工作流的同步锁配置
 Synchronization *Synchronization `json:"synchronization,omitempty" protobuf:"bytes,35,opt,name=synchronization,casttype=Synchronization"`

 // VolumeClaimGC describes the strategy to use when deleting volumes from completed workflows
 // 工作流程删除卷策略
 VolumeClaimGC *VolumeClaimGC `json:"volumeClaimGC,omitempty" protobuf:"bytes,36,opt,name=volumeClaimGC,casttype=VolumeClaimGC"`

 // RetryStrategy for all templates in the workflow.
 // 重试策略
 RetryStrategy *RetryStrategy `json:"retryStrategy,omitempty" protobuf:"bytes,37,opt,name=retryStrategy"`

 // PodMetadata defines additional metadata that should be applied to workflow pods
 // 植入额外的 labels 与 annotations 至工作流关联 POD
 PodMetadata *Metadata `json:"podMetadata,omitempty" protobuf:"bytes,38,opt,name=podMetadata"`

 // TemplateDefaults holds default template values that will apply to all templates in the Workflow, unless overridden on the template-level
 // 各项子任务的默认值
 TemplateDefaults *Template `json:"templateDefaults,omitempty" protobuf:"bytes,39,opt,name=templateDefaults"`

 // ArchiveLogs indicates if the container logs should be archived
 // 容器运行日志是否被归档保存
 ArchiveLogs *bool `json:"archiveLogs,omitempty" protobuf:"varint,40,opt,name=archiveLogs"`

 // Hooks holds the lifecycle hook which is invoked at lifecycle of
 // step, irrespective of the success, failure, or error status of the primary step
 // TODO;
 Hooks LifecycleHooks `json:"hooks,omitempty" protobuf:"bytes,41,opt,name=hooks"`

 // WorkflowMetadata contains some metadata of the workflow to refer to
 // 关联工作流的元数据
 WorkflowMetadata *WorkflowMetadata `json:"workflowMetadata,omitempty" protobuf:"bytes,42,opt,name=workflowMetadata"`

 // ArtifactGC describes the strategy to use when deleting artifacts from completed or deleted workflows (applies to all output Artifacts
 // unless Artifact.ArtifactGC is specified, which overrides this)
 // 删除工作流之后，附件的保留策略
 ArtifactGC *ArtifactGC `json:"artifactGC,omitempty" protobuf:"bytes,43,opt,name=artifactGC"`
}

Template

// Template is a reusable and composable unit of execution in a workflow
type Template struct {
 // Name is the name of the template
 Name string `json:"name,omitempty" protobuf:"bytes,1,opt,name=name"`

 // Inputs describe what inputs parameters and artifacts are supplied to this template
 Inputs Inputs `json:"inputs,omitempty" protobuf:"bytes,5,opt,name=inputs"`

 // Outputs describe the parameters and artifacts that this template produces
 Outputs Outputs `json:"outputs,omitempty" protobuf:"bytes,6,opt,name=outputs"`

 // NodeSelector is a selector to schedule this step of the workflow to be
 // run on the selected node(s). Overrides the selector set at the workflow level.
 NodeSelector map[string]string `json:"nodeSelector,omitempty" protobuf:"bytes,7,opt,name=nodeSelector"`

 // Affinity sets the pod's scheduling constraints
 // Overrides the affinity set at the workflow level (if any)
 Affinity *apiv1.Affinity `json:"affinity,omitempty" protobuf:"bytes,8,opt,name=affinity"`

 // Metdata sets the pods's metadata, i.e. annotations and labels
 Metadata Metadata `json:"metadata,omitempty" protobuf:"bytes,9,opt,name=metadata"`

 // Daemon will allow a workflow to proceed to the next step so long as the container reaches readiness
 Daemon *bool `json:"daemon,omitempty" protobuf:"bytes,10,opt,name=daemon"`

 // Steps define a series of sequential/parallel workflow steps
 Steps []ParallelSteps `json:"steps,omitempty" protobuf:"bytes,11,opt,name=steps"`

 // Container is the main container image to run in the pod
 Container *apiv1.Container `json:"container,omitempty" protobuf:"bytes,12,opt,name=container"`

 // ContainerSet groups multiple containers within a single pod.
 ContainerSet *ContainerSetTemplate `json:"containerSet,omitempty" protobuf:"bytes,40,opt,name=containerSet"`

 // Script runs a portion of code against an interpreter
 Script *ScriptTemplate `json:"script,omitempty" protobuf:"bytes,13,opt,name=script"`

 // Resource template subtype which can run k8s resources
 Resource *ResourceTemplate `json:"resource,omitempty" protobuf:"bytes,14,opt,name=resource"`

 // DAG template subtype which runs a DAG
 DAG *DAGTemplate `json:"dag,omitempty" protobuf:"bytes,15,opt,name=dag"`

 // Suspend template subtype which can suspend a workflow when reaching the step
 Suspend *SuspendTemplate `json:"suspend,omitempty" protobuf:"bytes,16,opt,name=suspend"`

 // Data is a data template
 Data *Data `json:"data,omitempty" protobuf:"bytes,39,opt,name=data"`

 // HTTP makes a HTTP request
 HTTP *HTTP `json:"http,omitempty" protobuf:"bytes,42,opt,name=http"`

 // Plugin is a plugin template
 Plugin *Plugin `json:"plugin,omitempty" protobuf:"bytes,43,opt,name=plugin"`

 // Volumes is a list of volumes that can be mounted by containers in a template.
 // +patchStrategy=merge
 // +patchMergeKey=name
 Volumes []apiv1.Volume `json:"volumes,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,17,opt,name=volumes"`

 // InitContainers is a list of containers which run before the main container.
 // +patchStrategy=merge
 // +patchMergeKey=name
 InitContainers []UserContainer `json:"initContainers,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,18,opt,name=initContainers"`

 // Sidecars is a list of containers which run alongside the main container
 // Sidecars are automatically killed when the main container completes
 // +patchStrategy=merge
 // +patchMergeKey=name
 Sidecars []UserContainer `json:"sidecars,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,19,opt,name=sidecars"`

 // Location in which all files related to the step will be stored (logs, artifacts, etc...).
 // Can be overridden by individual items in Outputs. If omitted, will use the default
 // artifact repository location configured in the controller, appended with the
 // <workflowname>/<nodename> in the key.
 ArchiveLocation *ArtifactLocation `json:"archiveLocation,omitempty" protobuf:"bytes,20,opt,name=archiveLocation"`

 // Optional duration in seconds relative to the StartTime that the pod may be active on a node
 // before the system actively tries to terminate the pod; value must be positive integer
 // This field is only applicable to container and script templates.
 ActiveDeadlineSeconds *intstr.IntOrString `json:"activeDeadlineSeconds,omitempty" protobuf:"bytes,21,opt,name=activeDeadlineSeconds"`

 // RetryStrategy describes how to retry a template when it fails
 RetryStrategy *RetryStrategy `json:"retryStrategy,omitempty" protobuf:"bytes,22,opt,name=retryStrategy"`

 // Parallelism limits the max total parallel pods that can execute at the same time within the
 // boundaries of this template invocation. If additional steps/dag templates are invoked, the
 // pods created by those templates will not be counted towards this total.
 Parallelism *int64 `json:"parallelism,omitempty" protobuf:"bytes,23,opt,name=parallelism"`

 // FailFast, if specified, will fail this template if any of its child pods has failed. This is useful for when this
 // template is expanded with `withItems`, etc.
 FailFast *bool `json:"failFast,omitempty" protobuf:"varint,41,opt,name=failFast"`

 // Tolerations to apply to workflow pods.
 // +patchStrategy=merge
 // +patchMergeKey=key
 Tolerations []apiv1.Toleration `json:"tolerations,omitempty" patchStrategy:"merge" patchMergeKey:"key" protobuf:"bytes,24,opt,name=tolerations"`

 // If specified, the pod will be dispatched by specified scheduler.
 // Or it will be dispatched by workflow scope scheduler if specified.
 // If neither specified, the pod will be dispatched by default scheduler.
 // +optional
 SchedulerName string `json:"schedulerName,omitempty" protobuf:"bytes,25,opt,name=schedulerName"`

 // PriorityClassName to apply to workflow pods.
 PriorityClassName string `json:"priorityClassName,omitempty" protobuf:"bytes,26,opt,name=priorityClassName"`

 // Priority to apply to workflow pods.
 Priority *int32 `json:"priority,omitempty" protobuf:"bytes,27,opt,name=priority"`

 // ServiceAccountName to apply to workflow pods
 ServiceAccountName string `json:"serviceAccountName,omitempty" protobuf:"bytes,28,opt,name=serviceAccountName"`

 // AutomountServiceAccountToken indicates whether a service account token should be automatically mounted in pods.
 // ServiceAccountName of ExecutorConfig must be specified if this value is false.
 AutomountServiceAccountToken *bool `json:"automountServiceAccountToken,omitempty" protobuf:"varint,32,opt,name=automountServiceAccountToken"`

 // Executor holds configurations of the executor container.
 Executor *ExecutorConfig `json:"executor,omitempty" protobuf:"bytes,33,opt,name=executor"`

 // HostAliases is an optional list of hosts and IPs that will be injected into the pod spec
 // +patchStrategy=merge
 // +patchMergeKey=ip
 HostAliases []apiv1.HostAlias `json:"hostAliases,omitempty" patchStrategy:"merge" patchMergeKey:"ip" protobuf:"bytes,29,opt,name=hostAliases"`

 // SecurityContext holds pod-level security attributes and common container settings.
 // Optional: Defaults to empty. See type description for default values of each field.
 // +optional
 SecurityContext *apiv1.PodSecurityContext `json:"securityContext,omitempty" protobuf:"bytes,30,opt,name=securityContext"`

 // PodSpecPatch holds strategic merge patch to apply against the pod spec. Allows parameterization of
 // container fields which are not strings (e.g. resource limits).
 PodSpecPatch string `json:"podSpecPatch,omitempty" protobuf:"bytes,31,opt,name=podSpecPatch"`

 // Metrics are a list of metrics emitted from this template
 Metrics *Metrics `json:"metrics,omitempty" protobuf:"bytes,35,opt,name=metrics"`

 // Synchronization holds synchronization lock configuration for this template
 Synchronization *Synchronization `json:"synchronization,omitempty" protobuf:"bytes,36,opt,name=synchronization,casttype=Synchronization"`

 // Memoize allows templates to use outputs generated from already executed templates
 Memoize *Memoize `json:"memoize,omitempty" protobuf:"bytes,37,opt,name=memoize"`

 // Timeout allows to set the total node execution timeout duration counting from the node's start time.
 // This duration also includes time in which the node spends in Pending state. This duration may not be applied to Step or DAG templates.
 Timeout string `json:"timeout,omitempty" protobuf:"bytes,38,opt,name=timeout"`
}

Inputs

// Inputs are the mechanism for passing parameters, artifacts, volumes from one template to another
type Inputs struct {
 // Parameters are a list of parameters passed as inputs
 // +patchStrategy=merge
 // +patchMergeKey=name
 Parameters []Parameter `json:"parameters,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,1,opt,name=parameters"`

 // Artifact are a list of artifacts passed as inputs
 // +patchStrategy=merge
 // +patchMergeKey=name
 Artifacts Artifacts `json:"artifacts,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,2,opt,name=artifacts"`
}

Outputs

// Outputs hold parameters, artifacts, and results from a step
type Outputs struct {
 // Parameters holds the list of output parameters produced by a step
 // +patchStrategy=merge
 // +patchMergeKey=name
 Parameters []Parameter `json:"parameters,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,1,rep,name=parameters"`

 // Artifacts holds the list of output artifacts produced by a step
 // +patchStrategy=merge
 // +patchMergeKey=name
 Artifacts Artifacts `json:"artifacts,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,2,rep,name=artifacts"`

 // Result holds the result (stdout) of a script template
 Result *string `json:"result,omitempty" protobuf:"bytes,3,opt,name=result"`

 // ExitCode holds the exit code of a script template
 ExitCode *string `json:"exitCode,omitempty" protobuf:"bytes,4,opt,name=exitCode"`
}

Metadata

// Pod metdata
type Metadata struct {
 Annotations map[string]string `json:"annotations,omitempty" protobuf:"bytes,1,opt,name=annotations"`
 Labels map[string]string `json:"labels,omitempty" protobuf:"bytes,2,opt,name=labels"`
}

ParallelSteps

// +kubebuilder:validation:Type=array
type ParallelSteps struct {
 Steps []WorkflowStep `json:"-" protobuf:"bytes,1,rep,name=steps"`
}

apiv1.Container

见 Pod Container

ContainerSetTemplate

type ContainerSetTemplate struct {
 Containers []ContainerNode `json:"containers" protobuf:"bytes,4,rep,name=containers"`
 VolumeMounts []corev1.VolumeMount `json:"volumeMounts,omitempty" protobuf:"bytes,3,rep,name=volumeMounts"`
 // RetryStrategy describes how to retry a container nodes in the container set if it fails.
 // Nbr of retries(default 0) and sleep duration between retries(default 0s, instant retry) can be set.
 RetryStrategy *ContainerSetRetryStrategy `json:"retryStrategy,omitempty" protobuf:"bytes,5,opt,name=retryStrategy"`
}

ScriptTemplate

// ScriptTemplate is a template subtype to enable scripting through code steps
type ScriptTemplate struct {
 apiv1.Container `json:",inline" protobuf:"bytes,1,opt,name=container"`

 // Source contains the source code of the script to execute
 Source string `json:"source" protobuf:"bytes,2,opt,name=source"`
}

ResourceTemplate

// ResourceTemplate is a template subtype to manipulate kubernetes resources
type ResourceTemplate struct {
 // Action is the action to perform to the resource.
 // Must be one of: get, create, apply, delete, replace, patch
 Action string `json:"action" protobuf:"bytes,1,opt,name=action"`

 // MergeStrategy is the strategy used to merge a patch. It defaults to "strategic"
 // Must be one of: strategic, merge, json
 MergeStrategy string `json:"mergeStrategy,omitempty" protobuf:"bytes,2,opt,name=mergeStrategy"`

 // Manifest contains the kubernetes manifest
 Manifest string `json:"manifest,omitempty" protobuf:"bytes,3,opt,name=manifest"`

 // ManifestFrom is the source for a single kubernetes manifest
 ManifestFrom *ManifestFrom `json:"manifestFrom,omitempty" protobuf:"bytes,8,opt,name=manifestFrom"`

 // SetOwnerReference sets the reference to the workflow on the OwnerReference of generated resource.
 SetOwnerReference bool `json:"setOwnerReference,omitempty" protobuf:"varint,4,opt,name=setOwnerReference"`

 // SuccessCondition is a label selector expression which describes the conditions
 // of the k8s resource in which it is acceptable to proceed to the following step
 SuccessCondition string `json:"successCondition,omitempty" protobuf:"bytes,5,opt,name=successCondition"`

 // FailureCondition is a label selector expression which describes the conditions
 // of the k8s resource in which the step was considered failed
 FailureCondition string `json:"failureCondition,omitempty" protobuf:"bytes,6,opt,name=failureCondition"`

 // Flags is a set of additional options passed to kubectl before submitting a resource
 // I.e. to disable resource validation:
 // flags: [
 // "--validate=false" # disable resource validation
 // ]
 Flags []string `json:"flags,omitempty" protobuf:"varint,7,opt,name=flags"`
}

DAGTemplate

// DAGTemplate is a template subtype for directed acyclic graph templates
type DAGTemplate struct {
 // Target are one or more names of targets to execute in a DAG
 Target string `json:"target,omitempty" protobuf:"bytes,1,opt,name=target"`

 // Tasks are a list of DAG tasks
 // +patchStrategy=merge
 // +patchMergeKey=name
 Tasks []DAGTask `json:"tasks" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,2,rep,name=tasks"`

 // This flag is for DAG logic. The DAG logic has a built-in "fail fast" feature to stop scheduling new steps,
 // as soon as it detects that one of the DAG nodes is failed. Then it waits until all DAG nodes are completed
 // before failing the DAG itself.
 // The FailFast flag default is true, if set to false, it will allow a DAG to run all branches of the DAG to
 // completion (either success or failure), regardless of the failed outcomes of branches in the DAG.
 // More info and example about this feature at https://github.com/argoproj/argo-workflows/issues/1442
 FailFast *bool `json:"failFast,omitempty" protobuf:"varint,3,opt,name=failFast"`
}

// DAGTask represents a node in the graph during DAG execution
type DAGTask struct {
 // Name is the name of the target
 Name string `json:"name" protobuf:"bytes,1,opt,name=name"`

 // Name of template to execute
 Template string `json:"template,omitempty" protobuf:"bytes,2,opt,name=template"`

 // Inline is the template. Template must be empty if this is declared (and vice-versa).
 Inline *Template `json:"inline,omitempty" protobuf:"bytes,14,opt,name=inline"`

 // Arguments are the parameter and artifact arguments to the template
 Arguments Arguments `json:"arguments,omitempty" protobuf:"bytes,3,opt,name=arguments"`

 // TemplateRef is the reference to the template resource to execute.
 TemplateRef *TemplateRef `json:"templateRef,omitempty" protobuf:"bytes,4,opt,name=templateRef"`

 // Dependencies are name of other targets which this depends on
 Dependencies []string `json:"dependencies,omitempty" protobuf:"bytes,5,rep,name=dependencies"`

 // WithItems expands a task into multiple parallel tasks from the items in the list
 WithItems []Item `json:"withItems,omitempty" protobuf:"bytes,6,rep,name=withItems"`

 // WithParam expands a task into multiple parallel tasks from the value in the parameter,
 // which is expected to be a JSON list.
 WithParam string `json:"withParam,omitempty" protobuf:"bytes,7,opt,name=withParam"`

 // WithSequence expands a task into a numeric sequence
 WithSequence *Sequence `json:"withSequence,omitempty" protobuf:"bytes,8,opt,name=withSequence"`

 // When is an expression in which the task should conditionally execute
 When string `json:"when,omitempty" protobuf:"bytes,9,opt,name=when"`

 // ContinueOn makes argo to proceed with the following step even if this step fails.
 // Errors and Failed states can be specified
 ContinueOn *ContinueOn `json:"continueOn,omitempty" protobuf:"bytes,10,opt,name=continueOn"`

 // OnExit is a template reference which is invoked at the end of the
 // template, irrespective of the success, failure, or error of the
 // primary template.
 // DEPRECATED: Use Hooks[exit].Template instead.
 OnExit string `json:"onExit,omitempty" protobuf:"bytes,11,opt,name=onExit"`

 // Depends are name of other targets which this depends on
 Depends string `json:"depends,omitempty" protobuf:"bytes,12,opt,name=depends"`

 // Hooks hold the lifecycle hook which is invoked at lifecycle of
 // task, irrespective of the success, failure, or error status of the primary task
 Hooks LifecycleHooks `json:"hooks,omitempty" protobuf:"bytes,13,opt,name=hooks"`
}

SuspendTemplate

// SuspendTemplate is a template subtype to suspend a workflow at a predetermined point in time
type SuspendTemplate struct {
 // Duration is the seconds to wait before automatically resuming a template. Must be a string. Default unit is seconds.
 // Could also be a Duration, e.g.: "2m", "6h", "1d"
 Duration string `json:"duration,omitempty" protobuf:"bytes,1,opt,name=duration"`
}

Data

// Data is a data template
type Data struct {
 // Source sources external data into a data template
 Source DataSource `json:"source" protobuf:"bytes,1,opt,name=source"`

 // Transformation applies a set of transformations
 Transformation Transformation `json:"transformation" protobuf:"bytes,2,rep,name=transformation"`
}

type Transformation []TransformationStep

type TransformationStep struct {
 // Expression defines an expr expression to apply
 Expression string `json:"expression" protobuf:"bytes,1,opt,name=expression"`
}

// DataSource sources external data into a data template
type DataSource struct {
 // ArtifactPaths is a data transformation that collects a list of artifact paths
 ArtifactPaths *ArtifactPaths `json:"artifactPaths,omitempty" protobuf:"bytes,1,opt,name=artifactPaths"`
}

// ArtifactPaths expands a step from a collection of artifacts
type ArtifactPaths struct {
 // Artifact is the artifact location from which to source the artifacts, it can be a directory
 Artifact `json:",inline" protobuf:"bytes,1,opt,name=artifact"`
}

HTTP

type HTTP struct {
 // Method is HTTP methods for HTTP Request
 Method string `json:"method,omitempty" protobuf:"bytes,1,opt,name=method"`
 // URL of the HTTP Request
 URL string `json:"url" protobuf:"bytes,2,opt,name=url"`
 // Headers are an optional list of headers to send with HTTP requests
 Headers HTTPHeaders `json:"headers,omitempty" protobuf:"bytes,3,rep,name=headers"`
 // TimeoutSeconds is request timeout for HTTP Request. Default is 30 seconds
 TimeoutSeconds *int64 `json:"timeoutSeconds,omitempty" protobuf:"bytes,4,opt,name=timeoutSeconds"`
 // SuccessCondition is an expression if evaluated to true is considered successful
 SuccessCondition string `json:"successCondition,omitempty" protobuf:"bytes,6,opt,name=successCondition"`
 // Body is content of the HTTP Request
 Body string `json:"body,omitempty" protobuf:"bytes,5,opt,name=body"`
 // BodyFrom is content of the HTTP Request as Bytes
 BodyFrom *HTTPBodySource `json:"bodyFrom,omitempty" protobuf:"bytes,8,opt,name=bodyFrom"`
 // InsecureSkipVerify is a bool when if set to true will skip TLS verification for the HTTP client
 InsecureSkipVerify bool `json:"insecureSkipVerify,omitempty" protobuf:"bytes,7,opt,name=insecureSkipVerify"`
}

type HTTPHeader struct {
 Name string `json:"name" protobuf:"bytes,1,opt,name=name"`
 Value string `json:"value,omitempty" protobuf:"bytes,2,opt,name=value"`
 ValueFrom *HTTPHeaderSource `json:"valueFrom,omitempty" protobuf:"bytes,3,opt,name=valueFrom"`
}

type HTTPHeaderSource struct {
 SecretKeyRef *v1.SecretKeySelector `json:"secretKeyRef,omitempty" protobuf:"bytes,1,opt,name=secretKeyRef"`
}

type HTTPHeaders []HTTPHeader

// HTTPBodySource contains the source of the HTTP body.
type HTTPBodySource struct {
 Bytes []byte `json:"bytes,omitempty" protobuf:"bytes,1,opt,name=bytes"`
}

Plugin

// Plugin is an Object with exactly one key
type Plugin struct {
 Object `json:",inline" protobuf:"bytes,1,opt,name=object"`
}

apiv1.Volume

见 POD

UserContainer

// UserContainer is a container specified by a user.
type UserContainer struct {
 apiv1.Container `json:",inline" protobuf:"bytes,1,opt,name=container"`

 // MirrorVolumeMounts will mount the same volumes specified in the main container
 // to the container (including artifacts), at the same mountPaths. This enables
 // dind daemon to partially see the same filesystem as the main container in
 // order to use features such as docker volume binding
 MirrorVolumeMounts *bool `json:"mirrorVolumeMounts,omitempty" protobuf:"varint,2,opt,name=mirrorVolumeMounts"`
}

ArtifactLocation

// ArtifactLocation describes a location for a single or multiple artifacts.
// It is used as single artifact in the context of inputs/outputs (e.g. outputs.artifacts.artname).
// It is also used to describe the location of multiple artifacts such as the archive location
// of a single workflow step, which the executor will use as a default location to store its files.
type ArtifactLocation struct {
 // ArchiveLogs indicates if the container logs should be archived
 ArchiveLogs *bool `json:"archiveLogs,omitempty" protobuf:"varint,1,opt,name=archiveLogs"`

 // S3 contains S3 artifact location details
 S3 *S3Artifact `json:"s3,omitempty" protobuf:"bytes,2,opt,name=s3"`

 // Git contains git artifact location details
 Git *GitArtifact `json:"git,omitempty" protobuf:"bytes,3,opt,name=git"`

 // HTTP contains HTTP artifact location details
 HTTP *HTTPArtifact `json:"http,omitempty" protobuf:"bytes,4,opt,name=http"`

 // Artifactory contains artifactory artifact location details
 Artifactory *ArtifactoryArtifact `json:"artifactory,omitempty" protobuf:"bytes,5,opt,name=artifactory"`

 // HDFS contains HDFS artifact location details
 HDFS *HDFSArtifact `json:"hdfs,omitempty" protobuf:"bytes,6,opt,name=hdfs"`

 // Raw contains raw artifact location details
 Raw *RawArtifact `json:"raw,omitempty" protobuf:"bytes,7,opt,name=raw"`

 // OSS contains OSS artifact location details
 OSS *OSSArtifact `json:"oss,omitempty" protobuf:"bytes,8,opt,name=oss"`

 // GCS contains GCS artifact location details
 GCS *GCSArtifact `json:"gcs,omitempty" protobuf:"bytes,9,opt,name=gcs"`

 // Azure contains Azure Storage artifact location details
 Azure *AzureArtifact `json:"azure,omitempty" protobuf:"bytes,10,opt,name=azure"`
}

RetryStrategy

// RetryStrategy provides controls on how to retry a workflow step
type RetryStrategy struct {
 // Limit is the maximum number of retry attempts when retrying a container. It does not include the original
 // container; the maximum number of total attempts will be `limit + 1`.
 Limit *intstr.IntOrString `json:"limit,omitempty" protobuf:"varint,1,opt,name=limit"`

 // RetryPolicy is a policy of NodePhase statuses that will be retried
 RetryPolicy RetryPolicy `json:"retryPolicy,omitempty" protobuf:"bytes,2,opt,name=retryPolicy,casttype=RetryPolicy"`

 // Backoff is a backoff strategy
 Backoff *Backoff `json:"backoff,omitempty" protobuf:"bytes,3,opt,name=backoff,casttype=Backoff"`

 // Affinity prevents running workflow's step on the same host
 Affinity *RetryAffinity `json:"affinity,omitempty" protobuf:"bytes,4,opt,name=affinity"`

 // Expression is a condition expression for when a node will be retried. If it evaluates to false, the node will not
 // be retried and the retry strategy will be ignored
 Expression string `json:"expression,omitempty" protobuf:"bytes,5,opt,name=expression"`
}

apiv1.Toleration

见 TODO；

ExecutorConfig

// ExecutorConfig holds configurations of an executor container.
type ExecutorConfig struct {
 // ServiceAccountName specifies the service account name of the executor container.
 ServiceAccountName string `json:"serviceAccountName,omitempty" protobuf:"bytes,1,opt,name=serviceAccountName"`
}

apiv1.HostAlias

见 TODO；

apiv1.PodSecurityContext

见 PodSecurityContext。

Metrics

const (
 MetricTypeGauge MetricType = "Gauge"
 MetricTypeHistogram MetricType = "Histogram"
 MetricTypeCounter MetricType = "Counter"
 MetricTypeUnknown MetricType = "Unknown"
)

// Metrics are a list of metrics emitted from a Workflow/Template
type Metrics struct {
 // Prometheus is a list of prometheus metrics to be emitted
 Prometheus []*Prometheus `json:"prometheus" protobuf:"bytes,1,rep,name=prometheus"`
}

// Prometheus is a prometheus metric to be emitted
type Prometheus struct {
 // Name is the name of the metric
 Name string `json:"name" protobuf:"bytes,1,opt,name=name"`
 // Labels is a list of metric labels
 Labels []*MetricLabel `json:"labels,omitempty" protobuf:"bytes,2,rep,name=labels"`
 // Help is a string that describes the metric
 Help string `json:"help" protobuf:"bytes,3,opt,name=help"`
 // When is a conditional statement that decides when to emit the metric
 When string `json:"when,omitempty" protobuf:"bytes,4,opt,name=when"`
 // Gauge is a gauge metric
 Gauge *Gauge `json:"gauge,omitempty" protobuf:"bytes,5,opt,name=gauge"`
 // Histogram is a histogram metric
 Histogram *Histogram `json:"histogram,omitempty" protobuf:"bytes,6,opt,name=histogram"`
 // Counter is a counter metric
 Counter *Counter `json:"counter,omitempty" protobuf:"bytes,7,opt,name=counter"`
}

Synchronization

// SynchronizationStatus stores the status of semaphore and mutex.
type SynchronizationStatus struct {
 // Semaphore stores this workflow's Semaphore holder details
 Semaphore *SemaphoreStatus `json:"semaphore,omitempty" protobuf:"bytes,1,opt,name=semaphore"`
 // Mutex stores this workflow's mutex holder details
 Mutex *MutexStatus `json:"mutex,omitempty" protobuf:"bytes,2,opt,name=mutex"`
}

Memoize

// Memoization enables caching for the Outputs of the template
type Memoize struct {
 // Key is the key to use as the caching key
 Key string `json:"key" protobuf:"bytes,1,opt,name=key"`
 // Cache sets and configures the kind of cache
 Cache *Cache `json:"cache" protobuf:"bytes,2,opt,name=cache"`
 // MaxAge is the maximum age (e.g. "180s", "24h") of an entry that is still considered valid. If an entry is older
 // than the MaxAge, it will be ignored.
 MaxAge string `json:"maxAge" protobuf:"bytes,3,opt,name=maxAge"`
}

// Cache is the configuration for the type of cache to be used
type Cache struct {
 // ConfigMap sets a ConfigMap-based cache
 ConfigMap *apiv1.ConfigMapKeySelector `json:"configMap" protobuf:"bytes,1,opt,name=configMap"`
}

Parameter

// Parameter indicate a passed string parameter to a service template with an optional default value
type Parameter struct {
 // Name is the parameter name
 Name string `json:"name" protobuf:"bytes,1,opt,name=name"`

 // Default is the default value to use for an input parameter if a value was not supplied
 Default *AnyString `json:"default,omitempty" protobuf:"bytes,2,opt,name=default"`

 // Value is the literal value to use for the parameter.
 // If specified in the context of an input parameter, the value takes precedence over any passed values
 Value *AnyString `json:"value,omitempty" protobuf:"bytes,3,opt,name=value"`

 // ValueFrom is the source for the output parameter's value
 ValueFrom *ValueFrom `json:"valueFrom,omitempty" protobuf:"bytes,4,opt,name=valueFrom"`

 // GlobalName exports an output parameter to the global scope, making it available as
 // '{{workflow.outputs.parameters.XXXX}} and in workflow.status.outputs.parameters
 GlobalName string `json:"globalName,omitempty" protobuf:"bytes,5,opt,name=globalName"`

 // Enum holds a list of string values to choose from, for the actual value of the parameter
 Enum []AnyString `json:"enum,omitempty" protobuf:"bytes,6,rep,name=enum"`

 // Description is the parameter description
 Description *AnyString `json:"description,omitempty" protobuf:"bytes,7,opt,name=description"`
}

Artifacts

type Artifacts []Artifact

// Artifact indicates an artifact to place at a specified path
type Artifact struct {
 // name of the artifact. must be unique within a template's inputs/outputs.
 Name string `json:"name" protobuf:"bytes,1,opt,name=name"`

 // Path is the container path to the artifact
 Path string `json:"path,omitempty" protobuf:"bytes,2,opt,name=path"`

 // mode bits to use on this file, must be a value between 0 and 0777
 // set when loading input artifacts.
 Mode *int32 `json:"mode,omitempty" protobuf:"varint,3,opt,name=mode"`

 // From allows an artifact to reference an artifact from a previous step
 From string `json:"from,omitempty" protobuf:"bytes,4,opt,name=from"`

 // ArtifactLocation contains the location of the artifact
 ArtifactLocation `json:",inline" protobuf:"bytes,5,opt,name=artifactLocation"`

 // GlobalName exports an output artifact to the global scope, making it available as
 // '{{workflow.outputs.artifacts.XXXX}} and in workflow.status.outputs.artifacts
 GlobalName string `json:"globalName,omitempty" protobuf:"bytes,6,opt,name=globalName"`

 // Archive controls how the artifact will be saved to the artifact repository.
 Archive *ArchiveStrategy `json:"archive,omitempty" protobuf:"bytes,7,opt,name=archive"`

 // Make Artifacts optional, if Artifacts doesn't generate or exist
 Optional bool `json:"optional,omitempty" protobuf:"varint,8,opt,name=optional"`

 // SubPath allows an artifact to be sourced from a subpath within the specified source
 SubPath string `json:"subPath,omitempty" protobuf:"bytes,9,opt,name=subPath"`

 // If mode is set, apply the permission recursively into the artifact if it is a folder
 RecurseMode bool `json:"recurseMode,omitempty" protobuf:"varint,10,opt,name=recurseMode"`

 // FromExpression, if defined, is evaluated to specify the value for the artifact
 FromExpression string `json:"fromExpression,omitempty" protobuf:"bytes,11,opt,name=fromExpression"`

 // ArtifactGC describes the strategy to use when to deleting an artifact from completed or deleted workflows
 ArtifactGC *ArtifactGC `json:"artifactGC,omitempty" protobuf:"bytes,12,opt,name=artifactGC"`

 // Has this been deleted?
 Deleted bool `json:"deleted,omitempty" protobuf:"varint,13,opt,name=deleted"`
}

WorkflowStep

// WorkflowStep is a reference to a template to execute in a series of step
type WorkflowStep struct {
 // Name of the step
 Name string `json:"name,omitempty" protobuf:"bytes,1,opt,name=name"`

 // Template is the name of the template to execute as the step
 Template string `json:"template,omitempty" protobuf:"bytes,2,opt,name=template"`

 // Inline is the template. Template must be empty if this is declared (and vice-versa).
 Inline *Template `json:"inline,omitempty" protobuf:"bytes,13,opt,name=inline"`

 // Arguments hold arguments to the template
 Arguments Arguments `json:"arguments,omitempty" protobuf:"bytes,3,opt,name=arguments"`

 // TemplateRef is the reference to the template resource to execute as the step.
 TemplateRef *TemplateRef `json:"templateRef,omitempty" protobuf:"bytes,4,opt,name=templateRef"`

 // WithItems expands a step into multiple parallel steps from the items in the list
 WithItems []Item `json:"withItems,omitempty" protobuf:"bytes,5,rep,name=withItems"`

 // WithParam expands a step into multiple parallel steps from the value in the parameter,
 // which is expected to be a JSON list.
 WithParam string `json:"withParam,omitempty" protobuf:"bytes,6,opt,name=withParam"`

 // WithSequence expands a step into a numeric sequence
 WithSequence *Sequence `json:"withSequence,omitempty" protobuf:"bytes,7,opt,name=withSequence"`

 // When is an expression in which the step should conditionally execute
 When string `json:"when,omitempty" protobuf:"bytes,8,opt,name=when"`

 // ContinueOn makes argo to proceed with the following step even if this step fails.
 // Errors and Failed states can be specified
 ContinueOn *ContinueOn `json:"continueOn,omitempty" protobuf:"bytes,9,opt,name=continueOn"`

 // OnExit is a template reference which is invoked at the end of the
 // template, irrespective of the success, failure, or error of the
 // primary template.
 // DEPRECATED: Use Hooks[exit].Template instead.
 OnExit string `json:"onExit,omitempty" protobuf:"bytes,11,opt,name=onExit"`

 // Hooks holds the lifecycle hook which is invoked at lifecycle of
 // step, irrespective of the success, failure, or error status of the primary step
 Hooks LifecycleHooks `json:"hooks,omitempty" protobuf:"bytes,12,opt,name=hooks"`
}

S3Artifact

// S3Artifact is the location of an S3 artifact
type S3Artifact struct {
 S3Bucket `json:",inline" protobuf:"bytes,1,opt,name=s3Bucket"`

 // Key is the key in the bucket where the artifact resides
 Key string `json:"key,omitempty" protobuf:"bytes,2,opt,name=key"`
}

// S3Bucket contains the access information required for interfacing with an S3 bucket
type S3Bucket struct {
 // Endpoint is the hostname of the bucket endpoint
 Endpoint string `json:"endpoint,omitempty" protobuf:"bytes,1,opt,name=endpoint"`

 // Bucket is the name of the bucket
 Bucket string `json:"bucket,omitempty" protobuf:"bytes,2,opt,name=bucket"`

 // Region contains the optional bucket region
 Region string `json:"region,omitempty" protobuf:"bytes,3,opt,name=region"`

 // Insecure will connect to the service with TLS
 Insecure *bool `json:"insecure,omitempty" protobuf:"varint,4,opt,name=insecure"`

 // AccessKeySecret is the secret selector to the bucket's access key
 AccessKeySecret *apiv1.SecretKeySelector `json:"accessKeySecret,omitempty" protobuf:"bytes,5,opt,name=accessKeySecret"`

 // SecretKeySecret is the secret selector to the bucket's secret key
 SecretKeySecret *apiv1.SecretKeySelector `json:"secretKeySecret,omitempty" protobuf:"bytes,6,opt,name=secretKeySecret"`

 // RoleARN is the Amazon Resource Name (ARN) of the role to assume.
 RoleARN string `json:"roleARN,omitempty" protobuf:"bytes,7,opt,name=roleARN"`

 // UseSDKCreds tells the driver to figure out credentials based on sdk defaults.
 UseSDKCreds bool `json:"useSDKCreds,omitempty" protobuf:"varint,8,opt,name=useSDKCreds"`

 // CreateBucketIfNotPresent tells the driver to attempt to create the S3 bucket for output artifacts, if it doesn't exist. Setting Enabled Encryption will apply either SSE-S3 to the bucket if KmsKeyId is not set or SSE-KMS if it is.
 CreateBucketIfNotPresent *CreateS3BucketOptions `json:"createBucketIfNotPresent,omitempty" protobuf:"bytes,9,opt,name=createBucketIfNotPresent"`

 EncryptionOptions *S3EncryptionOptions `json:"encryptionOptions,omitempty" protobuf:"bytes,10,opt,name=encryptionOptions"`
}

GitArtifact

// GitArtifact is the location of an git artifact
type GitArtifact struct {
 // Repo is the git repository
 Repo string `json:"repo" protobuf:"bytes,1,opt,name=repo"`

 // Revision is the git commit, tag, branch to checkout
 Revision string `json:"revision,omitempty" protobuf:"bytes,2,opt,name=revision"`

 // Depth specifies clones/fetches should be shallow and include the given
 // number of commits from the branch tip
 Depth *uint64 `json:"depth,omitempty" protobuf:"bytes,3,opt,name=depth"`

 // Fetch specifies a number of refs that should be fetched before checkout
 Fetch []string `json:"fetch,omitempty" protobuf:"bytes,4,rep,name=fetch"`

 // UsernameSecret is the secret selector to the repository username
 UsernameSecret *apiv1.SecretKeySelector `json:"usernameSecret,omitempty" protobuf:"bytes,5,opt,name=usernameSecret"`

 // PasswordSecret is the secret selector to the repository password
 PasswordSecret *apiv1.SecretKeySelector `json:"passwordSecret,omitempty" protobuf:"bytes,6,opt,name=passwordSecret"`

 // SSHPrivateKeySecret is the secret selector to the repository ssh private key
 SSHPrivateKeySecret *apiv1.SecretKeySelector `json:"sshPrivateKeySecret,omitempty" protobuf:"bytes,7,opt,name=sshPrivateKeySecret"`

 // InsecureIgnoreHostKey disables SSH strict host key checking during git clone
 InsecureIgnoreHostKey bool `json:"insecureIgnoreHostKey,omitempty" protobuf:"varint,8,opt,name=insecureIgnoreHostKey"`

 // DisableSubmodules disables submodules during git clone
 DisableSubmodules bool `json:"disableSubmodules,omitempty" protobuf:"varint,9,opt,name=disableSubmodules"`

 // SingleBranch enables single branch clone, using the `branch` parameter
 SingleBranch bool `json:"singleBranch,omitempty" protobuf:"varint,10,opt,name=singleBranch"`

 // Branch is the branch to fetch when `SingleBranch` is enabled
 Branch string `json:"branch,omitempty" protobuf:"bytes,11,opt,name=branch"`
}

HTTPArtifact

// HTTPArtifact allows a file served on HTTP to be placed as an input artifact in a container
type HTTPArtifact struct {
 // URL of the artifact
 URL string `json:"url" protobuf:"bytes,1,opt,name=url"`

 // Headers are an optional list of headers to send with HTTP requests for artifacts
 Headers []Header `json:"headers,omitempty" protobuf:"bytes,2,rep,name=headers"`

 // Auth contains information for client authentication
 Auth *HTTPAuth `json:"auth,omitempty" protobuf:"bytes,3,opt,name=auth"`
}

Arguments

// Arguments to a template
type Arguments struct {
 // Parameters is the list of parameters to pass to the template or workflow
 // +patchStrategy=merge
 // +patchMergeKey=name
 Parameters []Parameter `json:"parameters,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,1,rep,name=parameters"`

 // Artifacts is the list of artifacts to pass to the template or workflow
 // +patchStrategy=merge
 // +patchMergeKey=name
 Artifacts Artifacts `json:"artifacts,omitempty" patchStrategy:"merge" patchMergeKey:"name" protobuf:"bytes,2,rep,name=artifacts"`
}

DAGTask

// DAGTask represents a node in the graph during DAG execution
type DAGTask struct {
 // Name is the name of the target
 Name string `json:"name" protobuf:"bytes,1,opt,name=name"`

 // Name of template to execute
 Template string `json:"template,omitempty" protobuf:"bytes,2,opt,name=template"`

 // Inline is the template. Template must be empty if this is declared (and vice-versa).
 Inline *Template `json:"inline,omitempty" protobuf:"bytes,14,opt,name=inline"`

 // Arguments are the parameter and artifact arguments to the template
 Arguments Arguments `json:"arguments,omitempty" protobuf:"bytes,3,opt,name=arguments"`

 // TemplateRef is the reference to the template resource to execute.
 TemplateRef *TemplateRef `json:"templateRef,omitempty" protobuf:"bytes,4,opt,name=templateRef"`

 // Dependencies are name of other targets which this depends on
 Dependencies []string `json:"dependencies,omitempty" protobuf:"bytes,5,rep,name=dependencies"`

 // WithItems expands a task into multiple parallel tasks from the items in the list
 WithItems []Item `json:"withItems,omitempty" protobuf:"bytes,6,rep,name=withItems"`

 // WithParam expands a task into multiple parallel tasks from the value in the parameter,
 // which is expected to be a JSON list.
 WithParam string `json:"withParam,omitempty" protobuf:"bytes,7,opt,name=withParam"`

 // WithSequence expands a task into a numeric sequence
 WithSequence *Sequence `json:"withSequence,omitempty" protobuf:"bytes,8,opt,name=withSequence"`

 // When is an expression in which the task should conditionally execute
 When string `json:"when,omitempty" protobuf:"bytes,9,opt,name=when"`

 // ContinueOn makes argo to proceed with the following step even if this step fails.
 // Errors and Failed states can be specified
 ContinueOn *ContinueOn `json:"continueOn,omitempty" protobuf:"bytes,10,opt,name=continueOn"`

 // OnExit is a template reference which is invoked at the end of the
 // template, irrespective of the success, failure, or error of the
 // primary template.
 // DEPRECATED: Use Hooks[exit].Template instead.
 OnExit string `json:"onExit,omitempty" protobuf:"bytes,11,opt,name=onExit"`

 // Depends are name of other targets which this depends on
 Depends string `json:"depends,omitempty" protobuf:"bytes,12,opt,name=depends"`

 // Hooks hold the lifecycle hook which is invoked at lifecycle of
 // task, irrespective of the success, failure, or error status of the primary task
 Hooks LifecycleHooks `json:"hooks,omitempty" protobuf:"bytes,13,opt,name=hooks"`
}

ContainerNode

type ContainerNode struct {
 corev1.Container `json:",inline" protobuf:"bytes,1,opt,name=container"`
 Dependencies []string `json:"dependencies,omitempty" protobuf:"bytes,2,rep,name=dependencies"`
}

ArchiveStrategy

// ArchiveStrategy describes how to archive files/directory when saving artifacts
type ArchiveStrategy struct {
 Tar *TarStrategy `json:"tar,omitempty" protobuf:"bytes,1,opt,name=tar"`
 None *NoneStrategy `json:"none,omitempty" protobuf:"bytes,2,opt,name=none"`
 Zip *ZipStrategy `json:"zip,omitempty" protobuf:"bytes,3,opt,name=zip"`
}

// TarStrategy will tar and gzip the file or directory when saving
type TarStrategy struct {
 // CompressionLevel specifies the gzip compression level to use for the artifact.
 // Defaults to gzip.DefaultCompression.
 CompressionLevel *int32 `json:"compressionLevel,omitempty" protobuf:"varint,1,opt,name=compressionLevel"`
}

// ZipStrategy will unzip zipped input artifacts
type ZipStrategy struct{}

// NoneStrategy indicates to skip tar process and upload the files or directory tree as independent
// files. Note that if the artifact is a directory, the artifact driver must support the ability to
// save/load the directory appropriately.
type NoneStrategy struct{}

ValueFrom

// ValueFrom describes a location in which to obtain the value to a parameter
type ValueFrom struct {
 // Path in the container to retrieve an output parameter value from in container templates
 Path string `json:"path,omitempty" protobuf:"bytes,1,opt,name=path"`

 // JSONPath of a resource to retrieve an output parameter value from in resource templates
 JSONPath string `json:"jsonPath,omitempty" protobuf:"bytes,2,opt,name=jsonPath"`

 // JQFilter expression against the resource object in resource templates
 JQFilter string `json:"jqFilter,omitempty" protobuf:"bytes,3,opt,name=jqFilter"`

 // Selector (https://github.com/expr-lang/expr) that is evaluated against the event to get the value of the parameter. E.g. `payload.message`
 Event string `json:"event,omitempty" protobuf:"bytes,7,opt,name=event"`

 // Parameter reference to a step or dag task in which to retrieve an output parameter value from
 // (e.g. '{{steps.mystep.outputs.myparam}}')
 Parameter string `json:"parameter,omitempty" protobuf:"bytes,4,opt,name=parameter"`

 // Supplied value to be filled in directly, either through the CLI, API, etc.
 Supplied *SuppliedValueFrom `json:"supplied,omitempty" protobuf:"bytes,6,opt,name=supplied"`

 // ConfigMapKeyRef is configmap selector for input parameter configuration
 ConfigMapKeyRef *apiv1.ConfigMapKeySelector `json:"configMapKeyRef,omitempty" protobuf:"bytes,9,opt,name=configMapKeyRef"`

 // Default specifies a value to be used if retrieving the value from the specified source fails
 Default *AnyString `json:"default,omitempty" protobuf:"bytes,5,opt,name=default"`

 // Expression, if defined, is evaluated to specify the value for the parameter
 Expression string `json:"expression,omitempty" protobuf:"bytes,8,rep,name=expression"`
}

WorkflowTemplateRef

// WorkflowTemplateRef is a reference to a WorkflowTemplate resource.
type WorkflowTemplateRef struct {
 // Name is the resource name of the workflow template.
 Name string `json:"name,omitempty" protobuf:"bytes,1,opt,name=name"`
 // ClusterScope indicates the referred template is cluster scoped (i.e. a ClusterWorkflowTemplate).
 ClusterScope bool `json:"clusterScope,omitempty" protobuf:"varint,2,opt,name=clusterScope"`
}

PodGC

// PodGC describes how to delete completed pods as they complete
type PodGC struct {
 // Strategy is the strategy to use. One of "OnPodCompletion", "OnPodSuccess", "OnWorkflowCompletion", "OnWorkflowSuccess". If unset, does not delete Pods
 Strategy PodGCStrategy `json:"strategy,omitempty" protobuf:"bytes,1,opt,name=strategy,casttype=PodGCStrategy"`
 // LabelSelector is the label selector to check if the pods match the labels before being added to the pod GC queue.
 LabelSelector *metav1.LabelSelector `json:"labelSelector,omitempty" protobuf:"bytes,2,opt,name=labelSelector"`
 // DeleteDelayDuration specifies the duration before pods in the GC queue get deleted.
 DeleteDelayDuration *metav1.Duration `json:"deleteDelayDuration,omitempty" protobuf:"bytes,3,opt,name=deleteDelayDuration"`
}

PodGCStrategy

// PodGCStrategy is the strategy when to delete completed pods for GC.
type PodGCStrategy string

// PodGCStrategy
const (
 PodGCOnPodNone PodGCStrategy = ""
 PodGCOnPodCompletion PodGCStrategy = "OnPodCompletion"
 PodGCOnPodSuccess PodGCStrategy = "OnPodSuccess"
 PodGCOnWorkflowCompletion PodGCStrategy = "OnWorkflowCompletion"
 PodGCOnWorkflowSuccess PodGCStrategy = "OnWorkflowSuccess"
)

TTLStrategy

// TTLStrategy is the strategy for the time to live depending on if the workflow succeeded or failed
type TTLStrategy struct {
 // SecondsAfterCompletion is the number of seconds to live after completion
 SecondsAfterCompletion *int32 `json:"secondsAfterCompletion,omitempty" protobuf:"bytes,1,opt,name=secondsAfterCompletion"`
 // SecondsAfterSuccess is the number of seconds to live after success
 SecondsAfterSuccess *int32 `json:"secondsAfterSuccess,omitempty" protobuf:"bytes,2,opt,name=secondsAfterSuccess"`
 // SecondsAfterFailure is the number of seconds to live after failure
 SecondsAfterFailure *int32 `json:"secondsAfterFailure,omitempty" protobuf:"bytes,3,opt,name=secondsAfterFailure"`
}

ArtifactGC

// ArtifactGCStrategy is the strategy when to delete artifacts for GC.
type ArtifactGCStrategy string

// ArtifactGCStrategy
const (
 ArtifactGCOnWorkflowCompletion ArtifactGCStrategy = "OnWorkflowCompletion"
 ArtifactGCOnWorkflowDeletion ArtifactGCStrategy = "OnWorkflowDeletion"
 ArtifactGCNever ArtifactGCStrategy = "Never"
 ArtifactGCStrategyUndefined ArtifactGCStrategy = ""
)

// ArtifactGC describes how to delete artifacts from completed Workflows - this is embedded into the WorkflowLevelArtifactGC, and also used for individual Artifacts to override that as needed
type ArtifactGC struct {
 // Strategy is the strategy to use.
 // +kubebuilder:validation:Enum="";OnWorkflowCompletion;OnWorkflowDeletion;Never
 Strategy ArtifactGCStrategy `json:"strategy,omitempty" protobuf:"bytes,1,opt,name=strategy,casttype=ArtifactGCStategy"`

 // PodMetadata is an optional field for specifying the Labels and Annotations that should be assigned to the Pod doing the deletion
 PodMetadata *Metadata `json:"podMetadata,omitempty" protobuf:"bytes,2,opt,name=podMetadata"`

 // ServiceAccountName is an optional field for specifying the Service Account that should be assigned to the Pod doing the deletion
 ServiceAccountName string `json:"serviceAccountName,omitempty" protobuf:"bytes,3,opt,name=serviceAccountName"`
}

Docs: 启动配置

Wed, 06 Nov 2024 22:29:59 +0800

简要概述

用于 “workflow-controller” 应用的配置。

配置示例

在 cli 启动配置

spec:
 selector:
 matchLabels:
 app: argo-server
 template:
 metadata:
 labels:
 app: argo-server
 spec:
 containers:
 - args:
 - server
 - "--auth-mode=sso"
 - "--secure=false"
 env: []
 image: quay.io/argoproj/argocli:latest
 name: argo-server

在 configmap 中配置

# This file describes the config settings available in the workflow controller configmap
apiVersion: v1
kind: ConfigMap
metadata:
 name: workflow-controller-configmap
 namespace: argo
data:
 # 在 2.7+ 版本之后，可以不需要 "config: |" 这个 key 包含了。
 #
 # instanceID is a label selector to limit the controller's watch to a specific instance. It
 # contains an arbitrary value that is carried forward into its pod labels, under the key
 # workflows.argoproj.io/controller-instanceid, for the purposes of workflow segregation. This
 # enables a controller to only receive workflow and pod events that it is interested about,
 # in order to support multiple controllers in a single cluster, and ultimately allows the
 # controller itself to be bundled as part of a higher level application. If omitted, the
 # controller watches workflows and pods that *are not* labeled with an instance id.
 # 在单 k8s 集群上多 argo 实例下区分各实例监听的资源
 instanceID: my-ci-controller

 # Namespace is a label selector filter to limit the controller's watch to a specific namespace
 namespace: my-namespace

 # Parallelism limits the max total parallel workflows that can execute at the same time
 # (available since Argo v2.3). Controller must be restarted to take effect.
 parallelism: "10"

 # Limit the maximum number of incomplete workflows in a namespace.
 # Intended for cluster installs that are multi-tenancy environments, to prevent too many workflows in one
 # namespace impacting others.
 # >= v3.2
 namespaceParallelism: "10"

 # Globally limits the rate at which pods are created.
 # This is intended to mitigate flooding of the Kubernetes API server by workflows with a large amount of
 # parallel nodes.
 resourceRateLimit: |
 limit: 10
 burst: 1

 # Whether or not to emit events on node completion. These can take a up a lot of space in
 # k8s (typically etcd) resulting in errors when trying to create new events:
 # "Unable to create audit event: etcdserver: mvcc: database space exceeded"
 # This config item allows you to disable this.
 # (since v2.9)
 nodeEvents: |
  enabled: true

 # Whether or not to emit events on workflow status changes. These can take a up a lot of space in
 # k8s (typically etcd), see nodeEvents above.
 # This config item allows you to disable this.
 # (since v3.6)
 workflowEvents: |
  enabled: true

 # uncomment following lines if workflow controller runs in a different k8s cluster with the
 # workflow workloads, or needs to communicate with the k8s apiserver using an out-of-cluster
 # kubeconfig secret
 # kubeConfig:
 # # name of the kubeconfig secret, may not be empty when kubeConfig specified
 # secretName: kubeconfig-secret
 # # key of the kubeconfig secret, may not be empty when kubeConfig specified
 # secretKey: kubeconfig
 # # mounting path of the kubeconfig secret, default to /kube/config
 # mountPath: /kubeconfig/mount/path
 # # volume name when mounting the secret, default to kubeconfig
 # volumeName: kube-config-volume

 links: |
 # Adds a button to the workflow page. E.g. linking to you logging facility.
 - name: Example Workflow Link
 scope: workflow
 url: http://logging-facility?namespace=${metadata.namespace}&workflowName=${metadata.name}&startedAt=${status.startedAt}&finishedAt=${status.finishedAt}
 # Adds a button next to the pod. E.g. linking to you logging facility but for the pod only.
 - name: Example Pod Link
 scope: pod
 url: http://logging-facility?namespace=${metadata.namespace}&podName=${metadata.name}&startedAt=${status.startedAt}&finishedAt=${status.finishedAt}
 - name: Pod Logs
 scope: pod-logs
 url: http://logging-facility?namespace=${metadata.namespace}&podName=${metadata.name}&startedAt=${status.startedAt}&finishedAt=${status.finishedAt}
 - name: Event Source Logs
 scope: event-source-logs
 url: http://logging-facility?namespace=${metadata.namespace}&podName=${metadata.name}&startedAt=${status.startedAt}&finishedAt=${status.finishedAt}
 - name: Sensor Logs
 scope: sensor-logs
 url: http://logging-facility?namespace=${metadata.namespace}&podName=${metadata.name}&startedAt=${status.startedAt}&finishedAt=${status.finishedAt}
 # Adds a button to the bottom right of every page to link to your organisation help or chat.
 - name: Get help
 scope: chat
 url: http://my-chat
 # Adds a button to the top of workflow view to navigate to customized views.
 - name: Completed Workflows
 scope: workflow-list
 url: http://workflows?label=workflows.argoproj.io/completed=true

 # Columns are custom columns that will be exposed in the Workflow List View.
 # (available since Argo v3.5)
 columns: |
 # Adds a column to the Workflow List View
 - # The name of this column, e.g., "Workflow Completed".
 name: Workflow Completed
 # The type of this column, "label" or "annotation".
 type: label
 # The key of the label or annotation, e.g., "workflows.argoproj.io/completed".
 key: workflows.argoproj.io/completed

 # uncomment following lines if you want to change navigation bar background color
 # navColor: red

 # artifactRepository defines the default location to be used as the artifact repository for
 # container artifacts.
 # 工作 pod 生成的制品存放仓库
 artifactRepository: |
 # archiveLogs will archive the main container logs as an artifact
 # 各 pod 执行的日志是否归档至仓库
 archiveLogs: true

 s3:
 # Use the corresponding endpoint depending on your S3 provider:
 # AWS: s3.amazonaws.com
 # GCS: storage.googleapis.com
 # Minio: my-minio-endpoint.default:9000
 endpoint: s3.amazonaws.com
 bucket: my-bucket
 region: us-west-2
 # insecure will disable TLS. Primarily used for minio installs not configured with TLS
 insecure: false
 # keyFormat is a format pattern to define how artifacts will be organized in a bucket.
 # It can reference workflow metadata variables such as workflow.namespace, workflow.name,
 # pod.name. Can also use strftime formating of workflow.creationTimestamp so that workflow
 # artifacts can be organized by date. If omitted, will use `{{workflow.name}}/{{pod.name}}`,
 # which has potential for have collisions.
 # The following example pattern organizes workflow artifacts under a "my-artifacts" sub dir,
 # then sub dirs for year, month, date and finally workflow name and pod.
 # e.g.: my-artifacts/2018/08/23/my-workflow-abc123/my-workflow-abc123-1234567890
 keyFormat: "my-artifacts\
 /{{workflow.creationTimestamp.Y}}\
 /{{workflow.creationTimestamp.m}}\
 /{{workflow.creationTimestamp.d}}\
 /{{workflow.name}}\
 /{{pod.name}}"
 # The actual secret object (in this example my-s3-credentials), should be created in every
 # namespace where a workflow needs to store its artifacts to S3. If omitted,
 # attempts to use IAM role to access the bucket (instead of accessKey/secretKey).
 accessKeySecret:
 name: my-s3-credentials
 key: accessKey
 secretKeySecret:
 name: my-s3-credentials
 key: secretKey
 # If this is set to true, argo workflows will use AWS SDK default credentials provider chain. This will allow things like
 # IRSA and any of the authentication methods that the golang SDK uses in it's default chain.
 # If you are using IRSA on AWS, and set this option to true, you will also need to modify Argo-Server Deployment with
 # `spec.template.spec.securityContext.fsGroup: 65534` configuration. This is required for IRSA to be able to access
 # `/var/run/secrets/eks.amazonaws.com/serviceaccount/token` file, and authenticate with AWS.
 useSDKCreds: false

 encryptionOptions:
 # If this is set to true, SSE-S3 encryption will be used to store objects
 # unless kmsKeyId or serverSideCustomerKeySecret is set
 enableEncryption: false
 # A valid kms key id. If this value is set, the object stored in s3 will be encrypted with SSE-KMS
 # Note: You cannot set both kmsKeyId and serverSideCustomerKeySecret
 # kmsKeyId: ''
 # Allows you to set a json blob of simple key value pairs. See
 # https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context
 # for more information
 # kmsEncryptionContext: ''
 # The actual secret object (in this example my-s3-credentials),
 # should be created when using a custom secret to encrypt objects in using SSE-C
 # Note: You cannot set both kmsKeyId and serverSideCustomerKeySecret
 # serverSideCustomerKeySecret:
 # name: my-s3-credentials
 # key: secretKey

 # The command/args for each image, needed when the command is not specified and the emissary executor is used.
 # https://argo-workflows.readthedocs.io/en/latest/workflow-executors/#emissary-emissary
 images: |
 argoproj/argosay:v2:
 cmd: [/argosay]
 docker/whalesay:latest:
 cmd: [/bin/bash]

 # Defaults for main containers. These can be overridden by the template.
 # <= v3.3 only `resources` are supported.
 # >= v3.4 all fields are supported, including security context.
 mainContainer: |
 imagePullPolicy: IfNotPresent
 resources:
 requests:
 cpu: 0.1
 memory: 64Mi
 limits:
 cpu: 0.5
 memory: 512Mi
 securityContext:
 allowPrivilegeEscalation: false
 capabilities:
 drop:
 - ALL
 readOnlyRootFilesystem: true
 runAsNonRoot: true
 runAsUser: 1000

 # executor controls how the init and wait container should be customized
 # (available since Argo v2.3)
 executor: |
 # 在启动 pod init 阶段执行的容器，自定义
 # image: quay.io/argoproj/argoexec:v3.5.7
 image: registry.cn-hangzhou.aliyuncs.com/kube-image-repo/argoexec:v3.5.7
 imagePullPolicy: IfNotPresent
 resources:
 requests:
 cpu: 0.1
 memory: 64Mi
 limits:
 cpu: 0.5
 memory: 512Mi
 # args & env allows command line arguments and environment variables to be appended to the
 # executor container and is mainly used for development/debugging purposes.
 args:
 - --loglevel
 - debug
 - --gloglevel
 - "6"
 env:
 # ARGO_TRACE enables some tracing information for debugging purposes. Currently it enables
 # logging of S3 request/response payloads (including auth headers)
 - name: ARGO_TRACE
 value: "1"

 # metricsConfig controls the path and port for prometheus metrics. Metrics are enabled and emitted on localhost:9090/metrics
 # by default.
 metricsConfig: |
 # Enabled controls metric emission. Default is true, set "enabled: false" to turn off
 enabled: true
 # Path is the path where metrics are emitted. Must start with a "/". Default is "/metrics"
 path: /metrics
 # Port is the port where metrics are emitted. Default is "9090"
 port: 8080
 # MetricsTTL sets how often custom metrics are cleared from memory. Default is "0", metrics are never cleared. Histogram metrics are never cleared.
 metricsTTL: "10m"
 # IgnoreErrors is a flag that instructs prometheus to ignore metric emission errors. Default is "false"
 ignoreErrors: false
 # Use a self-signed cert for TLS
 # >= 3.6: default true
 secure: true
 # Options for configuring individual metrics
 options:
 pod_missing:
 disable: true
 cronworkflows_triggered_total:
 disabledAttributes:
 - name
 k8s_request_duration:
 histogramBuckets: [ 1.0, 2.0, 10.0 ]
 # >= 3.6. Which temporality to use for OpenTelemetry. Default is "Cumulative"
 temporality: Delta

 # DEPRECATED: Legacy metrics are now removed, this field is ignored
 disableLegacy: false

 # telemetryConfig controls the path and port for prometheus telemetry. Telemetry is enabled and emitted in the same endpoint
 # as metrics by default, but can be overridden using this config.
 telemetryConfig: |
 enabled: true
 path: /telemetry
 port: 8080
 secure: true # Use a self-signed cert for TLS, default false

 # enable persistence using postgres
 persistence: |
 connectionPool:
 maxIdleConns: 100
 maxOpenConns: 0
 connMaxLifetime: 0s # 0 means connections don't have a max lifetime
 # if true node status is only saved to the persistence DB to avoid the 1MB limit in etcd
 nodeStatusOffLoad: false
 # save completed workloads to the workflow archive
 archive: false
 # the number of days to keep archived workflows (the default is forever)
 archiveTTL: 180d
 # skip database migration if needed.
 # skipMigration: true

 # LabelSelector determines the workflow that matches with the matchlabels or matchrequirements, will be archived.
 # https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
 archiveLabelSelector:
 matchLabels:
 workflows.argoproj.io/archive-strategy: "always"

 # Optional name of the cluster I'm running in. This must be unique for your cluster.
 clusterName: default
 postgresql:
 host: localhost
 port: 5432
 database: postgres
 tableName: argo_workflows
 # the database secrets must be in the same namespace of the controller
 userNameSecret:
 name: argo-postgres-config
 key: username
 passwordSecret:
 name: argo-postgres-config
 key: password
 ssl: true
 # sslMode must be one of: disable, require, verify-ca, verify-full
 # you can find more information about those ssl options here: https://godoc.org/github.com/lib/pq
 sslMode: require

 # Optional config for mysql:
 # mysql:
 # host: localhost
 # port: 3306
 # database: argo
 # tableName: argo_workflows
 # userNameSecret:
 # name: argo-mysql-config
 # key: username
 # passwordSecret:
 # name: argo-mysql-config
 # key: password

 # PodSpecLogStrategy enables the logging of pod specs in the controller log.
 # podSpecLogStrategy: |
 # failedPod: true
 # allPods: false

 # PodGCGracePeriodSeconds specifies the duration in seconds before a terminating pod is forcefully killed.
 # Value must be non-negative integer. A zero value indicates that the pod will be forcefully terminated immediately.
 # Defaults to the Kubernetes default of 30 seconds.
 podGCGracePeriodSeconds: "60"

 # PodGCDeleteDelayDuration specifies the duration before pods in the GC queue get deleted.
 # Value must be non-negative. A zero value indicates that the pods will be deleted immediately.
 # Defaults to 5 seconds.
 podGCDeleteDelayDuration: 30s

 # adds initial delay (for K8S clusters with mutating webhooks) to prevent workflow getting modified by MWC.
 # initialDelay: 5s

 # Workflow retention by number of workflows
 # retentionPolicy: |
 # completed: 10
 # failed: 3
 # errored: 3

 # Default values that will apply to all Workflows from this controller, unless overridden on the Workflow-level
 # See more: docs/default-workflow-specs.md
 workflowDefaults: |
 metadata:
 annotations:
 argo: workflows
 labels:
 foo: bar
 spec:
 ttlStrategy:
 secondsAfterSuccess: 5
 parallelism: 3

 # SSO Configuration for the Argo server.
 # You must also start argo server with `--auth-mode sso`.
 # https://argo-workflows.readthedocs.io/en/latest/argo-server-auth-mode/
 sso: |
 # This is the root URL of the OIDC provider (required).
 issuer: https://issuer.root.url/
 # Some OIDC providers have alternate root URLs that can be included. These should be reviewed carefully. (optional)
 issuerAlias: https://altissuer.root.url
 # This defines how long your login is valid for (in hours). (optional)
 # If omitted, defaults to 10h. Example below is 10 days.
 sessionExpiry: 240h
 # This is name of the secret and the key in it that contain OIDC client
 # ID issued to the application by the provider (required).
 clientId:
 name: client-id-secret
 key: client-id-key
 # This is name of the secret and the key in it that contain OIDC client
 # secret issued to the application by the provider (required).
 clientSecret:
 name: client-secret-secret
 key: client-secret-key
 # This is the redirect URL supplied to the provider (optional). It must
 # be in the form <argo-server-root-url>/oauth2/callback. It must be
 # browser-accessible. If omitted, will be automatically generated.
 redirectUrl: https://argo-server/oauth2/callback
 # Additional scopes to request. Typically needed for SSO RBAC. >= v2.12
 scopes:
 - groups
 - email
 - profile
 # RBAC Config. >= v2.12
 rbac:
 enabled: false
 # Skip TLS verify, not recommended in production environments. Useful for testing purposes. >= v3.2.4
 insecureSkipVerify: false

 # workflowRestrictions restricts the Workflows that the controller will process.
 # Current options:
 # Strict: Only Workflows using "workflowTemplateRef" will be processed. This allows the administrator of the controller
 # to set a "library" of templates that may be run by its operator, limiting arbitrary Workflow execution.
 # Secure: Only Workflows using "workflowTemplateRef" will be processed and the controller will enforce
 # that the WorkflowTemplate that is referenced hasn't changed between operations. If you want to make sure the operator of the
 # Workflow cannot run an arbitrary Workflow, use this option.
 workflowRestrictions: |
  templateReferencing: Strict

数据结构

github.com/argoproj/argo-workflows/config/config.go

Config

// Config contains the configuration settings for the workflow controller
type Config struct {

 // NodeEvents configures how node events are emitted
 NodeEvents NodeEvents `json:"nodeEvents,omitempty"`

 // WorkflowEvents configures how workflow events are emitted
 WorkflowEvents WorkflowEvents `json:"workflowEvents,omitempty"`

 // Executor holds container customizations for the executor to use when running pods
 Executor *apiv1.Container `json:"executor,omitempty"`

 // MainContainer holds container customization for the main container
 MainContainer *apiv1.Container `json:"mainContainer,omitempty"`

 // KubeConfig specifies a kube config file for the wait & init containers
 KubeConfig *KubeConfig `json:"kubeConfig,omitempty"`

 // ArtifactRepository contains the default location of an artifact repository for container artifacts
 ArtifactRepository wfv1.ArtifactRepository `json:"artifactRepository,omitempty"`

 // Namespace is a label selector filter to limit the controller's watch to a specific namespace
 Namespace string `json:"namespace,omitempty"`

 // InstanceID is a label selector to limit the controller's watch to a specific instance. It
 // contains an arbitrary value that is carried forward into its pod labels, under the key
 // workflows.argoproj.io/controller-instanceid, for the purposes of workflow segregation. This
 // enables a controller to only receive workflow and pod events that it is interested about,
 // in order to support multiple controllers in a single cluster, and ultimately allows the
 // controller itself to be bundled as part of a higher level application. If omitted, the
 // controller watches workflows and pods that *are not* labeled with an instance id.
 InstanceID string `json:"instanceID,omitempty"`

 // MetricsConfig specifies configuration for metrics emission. Metrics are enabled and emitted on localhost:9090/metrics
 // by default.
 MetricsConfig MetricsConfig `json:"metricsConfig,omitempty"`

 // TelemetryConfig specifies configuration for telemetry emission. Telemetry is enabled and emitted in the same endpoint
 // as metrics by default, but can be overridden using this config.
 TelemetryConfig MetricsConfig `json:"telemetryConfig,omitempty"`

 // Parallelism limits the max total parallel workflows that can execute at the same time
 Parallelism int `json:"parallelism,omitempty"`

 // NamespaceParallelism limits the max workflows that can execute at the same time in a namespace
 NamespaceParallelism int `json:"namespaceParallelism,omitempty"`

 // ResourceRateLimit limits the rate at which pods are created
 ResourceRateLimit *ResourceRateLimit `json:"resourceRateLimit,omitempty"`

 // Persistence contains the workflow persistence DB configuration
 Persistence *PersistConfig `json:"persistence,omitempty"`

 // Links to related apps.
 Links []*wfv1.Link `json:"links,omitempty"`

 // Columns are custom columns that will be exposed in the Workflow List View.
 Columns []*wfv1.Column `json:"columns,omitempty"`

 // WorkflowDefaults are values that will apply to all Workflows from this controller, unless overridden on the Workflow-level
 WorkflowDefaults *wfv1.Workflow `json:"workflowDefaults,omitempty"`

 // PodSpecLogStrategy enables the logging of podspec on controller log.
 PodSpecLogStrategy PodSpecLogStrategy `json:"podSpecLogStrategy,omitempty"`

 // PodGCGracePeriodSeconds specifies the duration in seconds before a terminating pod is forcefully killed.
 // Value must be non-negative integer. A zero value indicates that the pod will be forcefully terminated immediately.
 // Defaults to the Kubernetes default of 30 seconds.
 PodGCGracePeriodSeconds *int64 `json:"podGCGracePeriodSeconds,omitempty"`

 // PodGCDeleteDelayDuration specifies the duration before pods in the GC queue get deleted.
 // Value must be non-negative. A zero value indicates that the pods will be deleted immediately.
 // Defaults to 5 seconds.
 PodGCDeleteDelayDuration *metav1.Duration `json:"podGCDeleteDelayDuration,omitempty"`

 // WorkflowRestrictions restricts the controller to executing Workflows that meet certain restrictions
 WorkflowRestrictions *WorkflowRestrictions `json:"workflowRestrictions,omitempty"`

 // Adds configurable initial delay (for K8S clusters with mutating webhooks) to prevent workflow getting modified by MWC.
 InitialDelay metav1.Duration `json:"initialDelay,omitempty"`

 // The command/args for each image, needed when the command is not specified and the emissary executor is used.
 // https://argo-workflows.readthedocs.io/en/latest/workflow-executors/#emissary-emissary
 Images map[string]Image `json:"images,omitempty"`

 // Workflow retention by number of workflows
 RetentionPolicy *RetentionPolicy `json:"retentionPolicy,omitempty"`

 // NavColor is an ui navigation bar background color
 NavColor string `json:"navColor,omitempty"`

 // SSO in settings for single-sign on
 SSO SSOConfig `json:"sso,omitempty"`
}

通过以下代码对配置的 configmap 转换为以上 Config 数据结构：

func parseConfigMap(cm *apiv1.ConfigMap, config *Config) error {
 // The key in the configmap to retrieve workflow configuration from.
 // Content encoding is expected to be YAML.
 rawConfig, ok := cm.Data["config"]
 if ok && len(cm.Data) != 1 {
 return fmt.Errorf("if you have an item in your config map named 'config', you must only have one item")
 }
 if !ok {
 for name, value := range cm.Data {
 if strings.Contains(value, "\n") {
 // this mucky code indents with two spaces
 rawConfig = rawConfig + name + ":\n " + strings.Join(strings.Split(strings.Trim(value, "\n"), "\n"), "\n ") + "\n"
 } else {
 rawConfig = rawConfig + name + ": " + value + "\n"
 }
 }
 }
 err := yaml.UnmarshalStrict([]byte(rawConfig), config)
 return err
}

NodeEvents

type NodeEvents struct {
 Enabled *bool `json:"enabled,omitempty"`
 SendAsPod bool `json:"sendAsPod,omitempty"`
}

WorkflowEvents

type WorkflowEvents struct {
 Enabled *bool `json:"enabled,omitempty"`
}

*apiv1.Container

见 Pod Container。

KubeConfig

// KubeConfig is used for wait & init sidecar containers to communicate with a k8s apiserver by a outofcluster method,
// it is used when the workflow controller is in a different cluster with the workflow workloads
type KubeConfig struct {
 // SecretName of the kubeconfig secret
 // may not be empty if kuebConfig specified
 SecretName string `json:"secretName"`
 // SecretKey of the kubeconfig in the secret
 // may not be empty if kubeConfig specified
 SecretKey string `json:"secretKey"`
 // VolumeName of kubeconfig, default to 'kubeconfig'
 VolumeName string `json:"volumeName,omitempty"`
 // MountPath of the kubeconfig secret, default to '/kube/config'
 MountPath string `json:"mountPath,omitempty"`
}

wfv1.ArtifactRepository

TODO;

MetricsConfig

type MetricsTemporality string

const (
 MetricsTemporalityCumulative MetricsTemporality = "Cumulative"
 MetricsTemporalityDelta MetricsTemporality = "Delta"
)

// MetricsConfig defines a config for a metrics server
type MetricsConfig struct {
 // Enabled controls metric emission. Default is true, set "enabled: false" to turn off
 Enabled *bool `json:"enabled,omitempty"`
 // DisableLegacy turns off legacy metrics
 // DEPRECATED: Legacy metrics are now removed, this field is ignored
 DisableLegacy bool `json:"disableLegacy,omitempty"`
 // MetricsTTL sets how often custom metrics are cleared from memory
 MetricsTTL TTL `json:"metricsTTL,omitempty"`
 // Path is the path where metrics are emitted. Must start with a "/". Default is "/metrics"
 Path string `json:"path,omitempty"`
 // Port is the port where metrics are emitted. Default is "9090"
 Port int `json:"port,omitempty"`
 // IgnoreErrors is a flag that instructs prometheus to ignore metric emission errors
 IgnoreErrors bool `json:"ignoreErrors,omitempty"`
 // Secure is a flag that starts the metrics servers using TLS, defaults to true
 Secure *bool `json:"secure,omitempty"`
 // Modifiers configure metrics by name
 Modifiers map[string]MetricModifier `json:"modifiers,omitempty"`
 // Temporality of the OpenTelemetry metrics.
 // Enum of Cumulative or Delta, defaulting to Cumulative.
 // No effect on Prometheus metrics, which are always Cumulative.
 Temporality MetricsTemporality `json:"temporality,omitempty"`
}

ResourceRateLimit

type ResourceRateLimit struct {
 Limit float64 `json:"limit"`
 Burst int `json:"burst"`
}

PersistConfig

type PersistConfig struct {
 NodeStatusOffload bool `json:"nodeStatusOffLoad,omitempty"`
 // Archive workflows to persistence.
 Archive bool `json:"archive,omitempty"`
 // ArchivelabelSelector holds LabelSelector to determine workflow persistence.
 ArchiveLabelSelector *metav1.LabelSelector `json:"archiveLabelSelector,omitempty"`
 // in days
 ArchiveTTL TTL `json:"archiveTTL,omitempty"`
 ClusterName string `json:"clusterName,omitempty"`
 ConnectionPool *ConnectionPool `json:"connectionPool,omitempty"`
 PostgreSQL *PostgreSQLConfig `json:"postgresql,omitempty"`
 MySQL *MySQLConfig `json:"mysql,omitempty"`
 SkipMigration bool `json:"skipMigration,omitempty"`
}

wfv1.Link

TODO;

wfv1.Column

TODO;

wfv1.Workflow

见 Workflow。

PodSpecLogStrategy

// PodSpecLogStrategy contains the configuration for logging the pod spec in controller log for debugging purpose
type PodSpecLogStrategy struct {
 FailedPod bool `json:"failedPod,omitempty"`
 AllPods bool `json:"allPods,omitempty"`
}

WorkflowRestrictions

type WorkflowRestrictions struct {
 TemplateReferencing TemplateReferencing `json:"templateReferencing,omitempty"`
}

type TemplateReferencing string

const (
 TemplateReferencingStrict TemplateReferencing = "Strict"
 TemplateReferencingSecure TemplateReferencing = "Secure"
)

Image

type Image struct {
 Entrypoint []string `json:"entrypoint,omitempty"`
 Cmd []string `json:"cmd,omitempty"`
}

RetentionPolicy

// Workflow retention by number of workflows
type RetentionPolicy struct {
 Completed int `json:"completed,omitempty"`
 Failed int `json:"failed,omitempty"`
 Errored int `json:"errored,omitempty"`
}

SSOConfig

type SSOConfig struct {
 Issuer string `json:"issuer"`
 IssuerAlias string `json:"issuerAlias,omitempty"`
 ClientID apiv1.SecretKeySelector `json:"clientId"`
 ClientSecret apiv1.SecretKeySelector `json:"clientSecret"`
 RedirectURL string `json:"redirectUrl"`
 RBAC *RBACConfig `json:"rbac,omitempty"`
 // additional scopes (on top of "openid")
 Scopes []string `json:"scopes,omitempty"`
 SessionExpiry metav1.Duration `json:"sessionExpiry,omitempty"`
 // customGroupClaimName will override the groups claim name
 CustomGroupClaimName string `json:"customGroupClaimName,omitempty"`
 UserInfoPath string `json:"userInfoPath,omitempty"`
 InsecureSkipVerify bool `json:"insecureSkipVerify,omitempty"`
 FilterGroupsRegex []string `json:"filterGroupsRegex,omitempty"`
}

RBACConfig

type RBACConfig struct {
 Enabled bool `json:"enabled,omitempty"`
}

Docs: 输入输出

Thu, 30 May 2024 16:33:51 +0800

简要概述

在工作流中参数的输入，结果的输出。

在工作流模版中定义参数，以提供任务运行时读取。

配置参数输入

在模版中定义

为任务输入参数可以在以下几个地方设置：

在 “spec” 中使用 “arguments.parameters” 设置；

数据结构见 WorkflowSpec。

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
 generateName: hello-world-
spec:
 entrypoint: main
 arguments:
 parameters:
 - name: message
 value: hello world from parameters
...

在 “spec.templates” 中使用 “inputs.parameters” 设置；

数据结构见 Template。

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
 generateName: hello-world-
spec:
 entrypoint: main
 templates:
 - name: main
 inputs:
 parameters:
 - name: "username"
 value: "mingqing"
 container:
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: ["echo user: {{ inputs.parameters.username }}"]

在 “spec.templates.steps” 使用 “arguments.parameters” 设置；

数据结构见 Template.ParallelSteps 下 WorkflowStep。

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
 generateName: hello-world-
spec:
 entrypoint: main
 templates:
 - name: main
 steps:
 - - name: test1
 template: echo
 arguments:
 parameters:
 - name: "username"
 value: "opsaid"

 - name: echo
 inputs:
 parameters:
 - name: "username"
 container:
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: ["echo user: {{ inputs.parameters.username }}"]

在 “spec.templates.dag” 中使用 “arguments.parameters” 设置。

Template.DAGTemplate 下 DAGTask

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
 generateName: hello-world-
spec:
 entrypoint: main
 templates:
 - name: main
 dag:
 - tasks:
 - name: test1
 template: echo
 arguments:
 parameters:
 - name: "username"
 value: "opsaid"

 - name: echo
 inputs:
 parameters:
 - name: "username"
 container:
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: ["echo user: {{ inputs.parameters.username }}"]

先前输出作为输入

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
 generateName: hello-world-
spec:
 entrypoint: main

 templates:
 - name: main
 dag:
 tasks:
 - name: task-a
 template: echo1
 - name: task-b
 template: echo2
 dependencies: [ task-a ]
 arguments:
 parameters:
 - name: hello
 value: "{{ tasks.task-a.outputs.parameters.message }}"

 - name: echo1
 outputs:
 parameters:
 - name: message
 valueFrom:
 path: "/tmp/a.txt"
 container:
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: [ "echo task-a >> /tmp/a.txt" ]

 - name: echo2
 inputs:
 parameters:
 - name: hello
 container:
 image: ccr.ccs.tencentyun.com/opsaid/minideb:bullseye
 command: [ sh, -c ]
 args: [ "echo {{ inputs.parameters.hello }}" ]

如何引用变量

在工作流中一些字段是可以使用变量，在运行时 argo 自动对其进行替换，变量引用格式为：

"{{ variables }}"

支持的变量见Workflow Variables

全局变量

变量名	示例	描述
workflow.name	hello-world-5x7vm	工作流名称
workflow.namespace	default	工作流运行命名空间
workflow.mainEntrypoint	main	工作流入口任务名称
workflow.serviceAccountName	default	运行工作流配置的 sa 需主动配置，否则为 ‘-’
workflow.uid	1652e654-4f0a-43ca-975f-c1f591d0cf39	UUID
workflow.parameters	[{“name”: “key1”, “value”: “val1”}]	获取定义在 “WorkflowSpec.Arguments” 下的参数
workflow.parameters.json	[{“name”: “key1”, “value”: “val1”}]	获取定义在 “WorkflowSpec.Arguments” 下的参数
workflow.outputs.parameters.{name}	TODO;	TODO;
workflow.outputs.artifacts.{name}	TODO;	TODO;
workflow.annotations	{appname:hello-world}	获取工作流配置的注解
workflow.labels	{appname:hello-world}	获取工作流配置的标签
workflow.creationTimestamp	2024-05-31T08:40:10Z	工作流创建的时间
workflow.priority	10	工作流优先级，需存在设置
workflow.duration	10.518252	工作流运行的持续时间，单位：秒
workflow.scheduledTime	2024-05-31T08:40:10Z	工作流被调度的时间，仅 CronWorkflow 类型有效

在 templates 中均可使用

变量名	描述
inputs.parameters	模板的所有输入参数都作为 JSON 字符串
inputs.parameters.	在模版中输入的参数
inputs.artifacts.	在模版中输入的附件
node.name	运行这个任务的 POD 名称，如：hello-world-dcntk

在 templates.steps 中使用

变量名	示例	描述
steps.name	TODO;	该任务名称
steps..id	TODO;	指定任务名称的唯一 ID
steps.{name}.id	TODO;	根据前面任务的名，获取唯一任务 ID
steps.{name}.ip	TODO;	需在该任务之前且为daemon的任务才可获取值
steps.{name}.status	Succeeded	获取前面任务的运行状态
steps.{name}.exitCode	0	获取前面任务的运行退出状态码
steps.{name}.startedAt	2024-05-31T06:05:10Z	获取前面任务的开始运行时间
steps.{name}.finishedAt	2024-05-31T06:05:10Z	获取前面任务的完成运行时间
steps.{name}.outputs.result	hello	获取前面任务完成时的标准输出
steps.{name}.outputs.parameters	TODO;	获取前面任务使用 “withItems” 或者 “withParams” 运行，将会以 JSON 格式输出
steps.{name}.outputs.artifacts	TODO;	获取前面任务的输出附件

在 templates.dag 中使用

变量名	示例	描述
tasks.name	TODO	任务名
tasks.{name}.id	TODO;	含义同 “templates.steps”
tasks.{name}.ip	TODO	含义同 “templates.steps”
tasks.{name}.status	TODO;	含义同 “templates.steps”
tasks.{name}.exitCode	TODO;	含义同 “templates.steps”
tasks.{name}.startedAt	TODO;	含义同 “templates.steps”
tasks.{name}.finishedAt	TODO;	含义同 “templates.steps”
tasks.{name}.outputs.result	TODO;	含义同 “templates.steps”
tasks.{name}.outputs.parameters	TODO;	含义同 “templates.steps”
tasks.{name}.outputs.artifacts	TODO;	含义同 “templates.steps”

Docs: 日志归档

Fri, 07 Mar 2025 13:42:35 +0800

简要概述

工作流运行默认不依赖数据库。

Argo 将工作流存储为 Kubernetes 资源，因为此类型资源默认必须小于 1MB，每个资源都包含每个节点的状态，这些状态存储在 “/status/nodes” 字段中。当这个字段超过 1MB 时，会尝试压缩节点状态并将其存储在 “/status/compressedNodes” 中。如果状态仍然过大，则会要求将其存储在数据库中。

或者因为审计等问题需要把已经完成的任务做归档以便后续查询，此时就需要配置上数据库（支持：mysql 或 postgres）。

当数据库异常时：

如果 “workflow-controller” 服务已在运行，则不影响后续任务执行；
如果 “workflow-controller” 重启则会无法启动，因为无法连接上数据库；

有两类数据需要归档：

任务运行的元数据，归档至数据库；
任务运行的日志与附件，归档至对象存储；

归档元数据

配置开启

apiVersion: v1
kind: ConfigMap
metadata:
 name: workflow-controller-configmap
 namespace: argo
data:
 ......

 # enable persistence using postgres
 persistence: |
 connectionPool:
 maxIdleConns: 100
 maxOpenConns: 0
 connMaxLifetime: 0s # 0 means connections don't have a max lifetime
 # if true node status is only saved to the persistence DB to avoid the 1MB limit in etcd
 nodeStatusOffLoad: false
 # save completed workloads to the workflow archive
 # 在工作流结束后把信息归档到数据库
 archive: true
 # the number of days to keep archived workflows (the default is forever)
 # 多少天之后移除已归档的信息
 archiveTTL: 180d
 # skip database migration if needed.
 # skipMigration: true

 # LabelSelector determines the workflow that matches with the matchlabels or matchrequirements, will be archived.
 # https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
 # 仅符合特定标签任务才做归档
 #archiveLabelSelector:
 # matchLabels:
 # workflows.argoproj.io/archive-strategy: "always"

 # Optional name of the cluster I'm running in. This must be unique for your cluster.
 # 改集群唯一标识，用于多 argo 集群下区分
 clusterName: default
 postgresql:
 host: localhost
 port: 5432
 database: argo
 tableName: argo_workflows
 # the database secrets must be in the same namespace of the controller
 userNameSecret:
 name: argo-database-config
 key: username
 passwordSecret:
 name: argo-database-config
 key: password
 ssl: false
 # sslMode must be one of: disable, require, verify-ca, verify-full
 # you can find more information about those ssl options here: https://godoc.org/github.com/lib/pq
 sslMode: disable

 # Optional config for mysql:
 # mysql:
 # host: localhost
 # port: 3306
 # database: argo
 # tableName: argo_workflows
 # userNameSecret:
 # name: argo-mysql-config
 # key: username
 # passwordSecret:
 # name: argo-mysql-config
 # key: password

 ......

数据库权限

apiVersion: v1
kind: Secret
metadata:
 name: argo-database-config
 namespace: argo
type: Opaque
data:
 username: dXB0aW1l
 password: dGVzdGtleTI=

create database argo;

数据库表结构

argo=> \dt
 List of relations
 Schema | Name | Type | Owner
--------+--------------------------------+-------+--------
 public | argo_archived_workflows | table | uptime
 public | argo_archived_workflows_labels | table | uptime
 public | schema_history | table | uptime
 public | workflows | table | uptime
(4 rows)

argo=>

argo=> \d argo_archived_workflows
 Table "public.argo_archived_workflows"
 Column | Type | Collation | Nullable | Default
-------------+-----------------------------+-----------+----------+-------------------
 uid | character varying(128) | | not null |
 name | character varying(256) | | not null |
 phase | character varying(25) | | not null |
 namespace | character varying(256) | | not null |
 workflow | json | | not null |
 startedat | timestamp without time zone | | not null | CURRENT_TIMESTAMP
 finishedat | timestamp without time zone | | not null | CURRENT_TIMESTAMP
 clustername | character varying(64) | | not null |
 instanceid | character varying(64) | | not null |
Indexes:
 "argo_archived_workflows_pkey" PRIMARY KEY, btree (clustername, uid)
 "argo_archived_workflows_i1" btree (clustername, instanceid, namespace)
 "argo_archived_workflows_i2" btree (clustername, instanceid, finishedat)
 "argo_archived_workflows_i3" btree (clustername, instanceid, name)
 "argo_archived_workflows_i4" btree (startedat)
Referenced by:
 TABLE "argo_archived_workflows_labels" CONSTRAINT "argo_archived_workflows_labels_clustername_uid_fkey" FOREIGN KEY (clustername, uid) REFERENCES argo_archived_workflows(clustername, uid) ON DELETE CASCADE

argo=>

argo=> \d argo_archived_workflows_labels
 Table "public.argo_archived_workflows_labels"
 Column | Type | Collation | Nullable | Default
-------------+------------------------+-----------+----------+---------
 clustername | character varying(64) | | not null |
 uid | character varying(128) | | not null |
 name | character varying(317) | | not null |
 value | character varying(63) | | not null |
Indexes:
 "argo_archived_workflows_labels_pkey" PRIMARY KEY, btree (clustername, uid, name)
 "argo_archived_workflows_labels_i1" btree (name, value)
Foreign-key constraints:
 "argo_archived_workflows_labels_clustername_uid_fkey" FOREIGN KEY (clustername, uid) REFERENCES argo_archived_workflows(clustername, uid) ON DELETE CASCADE

argo=>

argo=> \d schema_history
 Table "public.schema_history"
 Column | Type | Collation | Nullable | Default
----------------+---------+-----------+----------+---------
 schema_version | integer | | not null |

argo=>

argo=> \d workflows
 Table "public.workflows"
 Column | Type | Collation | Nullable | Default
-------------+-----------------------------+-----------+----------+-------------------
 uid | character varying(128) | | not null |
 namespace | character varying(256) | | not null |
 clustername | character varying(64) | | not null |
 version | character varying(64) | | not null |
 nodes | json | | not null |
 updatedat | timestamp without time zone | | not null | CURRENT_TIMESTAMP
Indexes:
 "workflows_pkey" PRIMARY KEY, btree (clustername, uid, version)
 "workflows_i1" btree (clustername, namespace, updatedat)

argo=>

归档日志与附件

配置开启

Docs: 权限验证

Tue, 19 Nov 2024 09:53:20 +0800

简要概述

用户在登录 UI 后，可以支持 RBAC 权限控制，否则默认登录成功没有权限隔离。

配置示例

应用 “argo-server” 启动参数

apiVersion: apps/v1
kind: Deployment
metadata:
 name: argo-server
spec:
 selector:
 matchLabels:
 app: argo-server
 template:
 metadata:
 labels:
 app: argo-server
 spec:
 containers:
 - args:
 - server
 - "--auth-mode=sso"
 - "--secure=false"
 env:
 - name: SSO_DELEGATE_RBAC_TO_NAMESPACE
 value: "true"
......

控制器 “workflow-controller” ConfigMap

添加关键 “rbac.enabled: true” 开启，如：

apiVersion: v1
kind: ConfigMap
metadata:
 name: workflow-controller-configmap
data:
 sso: |
 ......
 scopes:
 - groups
 - email
 rbac:
 enabled: true
 ......

创建 argo 空间 sa 账号

---
apiVersion: v1
kind: ServiceAccount
metadata:
 name: biz-test-uptime-readwrite-limq
 namespace: argo
 annotations:
 workflows.argoproj.io/rbac-rule: "email == 'admin@example.com'"
 workflows.argoproj.io/rbac-rule-precedence: "1"
secrets:
- name: biz-test-uptime-readwrite-limq

---
apiVersion: v1
kind: Secret
metadata:
 name: biz-test-uptime-readwrite-limq
 namespace: argo
 annotations:
 kubernetes.io/service-account.name: biz-test-uptime-readwrite-limq
type: kubernetes.io/service-account-token

当通过 UI 使用 OIDC 登录成功后，后端服务 “argo-server” 在对应的部署空间内查找 “ServiceAccount” 账号。

以上规则，当 JWT 中 “email” 属性值为 “admin@example.com” 则对应 “biz-test-uptime-readwrite-limq” 账号。

创建运行工作流空间角色绑定

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
 name: argo-readwrite
 namespace: biz-test-uptime
rules:
- apiGroups:
 - argoproj.io
 resources:
 - workflows
 - workflows/finalizers
 - workfloweventbindings
 - workfloweventbindings/finalizers
 - workflowtemplates
 - workflowtemplates/finalizers
 - cronworkflows
 - cronworkflows/finalizers
 - clusterworkflowtemplates
 - clusterworkflowtemplates/finalizers
 - workflowtaskresults
 - workflowtaskresults/finalizers
 verbs:
 - get
 - list
 - watch

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: argo-readwrite
 namespace: biz-test-uptime
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: Role
 name: argo-readwrite
subjects:
- kind: ServiceAccount
 name: biz-test-uptime-readwrite-limq
 namespace: argo