Multiple Pipeline

Drone supports configuring and orchestrating multiple pipelines. This is useful when you need to fan-out and distribute your build tasks across multiple machines to reduce build times, or to execute your build tasks across multiple platforms (e.g. amd64 and arm64).

Configuration

Multi-machine pipelines are configured as multiple yaml documents. Pipelines are spread across runners and execute in parallel. The below example configures two parallel pipelines executions. The overall build status is determined by the successful completion of both pipelines.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
kind: pipeline
type: docker
name: backend

steps:
- name: build
  image: golang
  commands:
  - go build
  - go test

---
kind: pipeline
type: docker
name: frontend

steps:
- name: build
  image: node
  commands:
  - npm install
  - npm test

Graph Execution

When you define multiple pipelines they are executed in parallel by default. You can optionally describe your pipelines as a directed acyclic graph.

In the below example we fan-out to execute the first two pipelines in parallel, and then once complete, we fan-in to execute the final pipeline:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
kind: pipeline
type: docker
name: backend

steps:
- name: build
  image: golang
  commands:
  - go build
  - go test

---
kind: pipeline
type: docker
name: frontend

steps:
- name: build
  image: node
  commands:
  - npm install
  - npm test

---
kind: pipeline
name: after

steps:
- name: notify
  image: plugins/slack
  settings:
    room: general
    webhook: https://...

depends_on:
- backend
- frontend

The above example is quite simple, however, you can use this syntax to create very complex execution flows.

Multi-Platform

You can also use multi-machine pipelines to execute workloads on different operating systems and architectures. Below is a simple example of a multi-pipeline configuration that executes pipelines on amd64 and arm64 architectures.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
kind: pipeline
type: docker
name: amd64

platform:
  arch: amd64

steps:
- name: build
  image: golang
  commands:
  - go build
  - go test

---
kind: pipeline
type: docker
name: arm

platform:
  arch: arm64

steps:
- name: build
  image: node
  commands:
  - npm install
  - npm test

...