Version 0.9.0
Language EN

Multi-Machine Pipelines

Drone supports multi-machine pipelines. This is useful when you need to fan-out and distribute your build tasks across multiple machines to reduce build times.

Please be warned that most projects do not require multi-machine pipelines. You should exhaust every other option, including parallel pipeline steps and more powerful machines. Multi-machine pipelines will increase complexity, and could actually increase execution times when improperly used.


Multi-machine pipelines are configured as multiple yaml documents. Pipelines are spread across agents and share nothing. The below example configures two parallel pipelines executions. The overall build status is determined by the successful completion of both pipelines.

kind: pipeline
name: frontend

steps:
- name: build
  image: node
  commands:
  - npm install
  - npm test

---
kind: pipeline
name: backend

steps:
- name: build
  image: golang
  commands:
  - go build
  - go test

services:
- name: redis
  image: redis

If you configure a multi-machine or multi-platform pipeline, you must provide a unique name for each pipeline in the yaml:

---
kind: pipeline
name: frontend

Dependencies

The top-level depends_on declaration defines a pipeline dependency graph and allows fan-in and fan-out behavior, spanning multiple machines. The pipeline does not execute until the dependent pipelines successfully complete.

In the below example, the first two pipelines execute in parallel. On successful completion a third pipeline is executed that sends a Slack notification indicating the workflow is complete.

kind: pipeline
name: frontend

steps:
- name: build
  image: node
  commands:
  - npm install
  - npm test

---
kind: pipeline
name: backend

steps:
- name: build
  image: golang
  commands:
  - go build
  - go test

services:
- name: redis
  image: redis

---
kind: pipeline
name: after

steps:
- name: notify
  image: plugins/slack
  settings:
    room: general
    webhook: https://...

depends_on:
- frontend
- backend

...

Cloning

A full git clone is executed for each pipeline to ensure access to the source code. In our previous example, cloning is not required in order to send the Slack notification, and can be disabled:

---
kind: pipeline
name: after

clone:
  disable: true

steps:
- name: notify
  image: plugins/slack
  settings:
    room: general
    webhook: https://...

depends_on:
- frontend
- backend

Caching

A common requirement is to share artifacts (compiled binaries, bundles, etc) when running multiple, dependent pipelines. This is not something Drone handles natively, but can be easily achieved with shell commands (i.e. upload and download to S3) or better yet can be handled with a custom plugin.

Triggers

The top-level triggers declaration defines conditions that must be met for pipeline execution. This can be useful when you want to limit pipeline execution by branch or build event.

This can also be useful if you want to execute a pipeline event when prior pipeline dependencies are in a failing state. Building on the previous example, we can configure the notification pipeline to always execute, despite failure of dependent pipelines.

---
kind: pipeline
name: after

steps:
- name: notify
  image: plugins/slack
  settings:
    room: general
    webhook: https://...

depends_on:
- frontend
- backend

trigger:
  branch:
  - master

  status:
  - success
  - failure

On This Page:

Getting Help

Enterprise Support
Real-time chat support from the developers that wrote the code.
Mailing List
Search for information in the mailing list archives, or post a question.