drone-cache

A Drone plugin for caching current workspace files between builds to reduce your build times. drone-cache is a small CLI program, written in Go without any external OS dependencies (such as tar, etc).

With drone-cache, you can provide your own cache key templates, specify archive format (tar, tar.gz, etc) and you can use popular object storage as storage for your cached files, even better you can implement your custom storage backend to cover your use case.

For detailed usage information and a list of available options please take a look at usage and examples. If you want to learn more about custom cache keys, see cache key templates.

If you want to learn more about the story behind drone-cache, you can read our blogpost Making Drone Builds 10 Times Faster!!

Git Leaks

Run the following script to install git-leaks support to this repo.

chmod +x ./git-hooks/install.sh
./git-hooks/install.sh

Supported Storage Backends

• AWS S3
  • Configuration
  • Example
  • Other AWS API compatible stores:
    • Minio
    • Red Hat Ceph
    • IBM Object Store
    • and many many others
• Azure Storage
  • Configuration
  • Example
• Google Cloud Storage
  • Configuration
  • Example
• or any mounted local volume
  • Configuration
  • Example

How does it work

drone-cache stores mounted directories and files under a key at the specified backend (by default S3).

Use this plugin to cache data that makes your builds faster. In the case of a cache miss or an empty cache restore it will fail silently in won't break your running pipeline.

The best example would be to use this with your package managers such as Mix, Bundler or Maven. After your initial download, you can build a cache and then you can restore that cache in your next build.

With restored dependencies from a cache, commands like mix deps.get will only need to download new dependencies, rather than re-download every package on each build.

Supported Build Tools

drone-cache automatically detects and configures cache directories for popular build tools. Configuration is injected into native tool config files, enabling caching to work seamlessly in containerized CI environments where only the repository directory is mounted.

Auto-Detection and Configuration

The plugin automatically detects the presence of lock/manifest files and configures the respective package manager to cache dependencies in repo-local directories (under .cache/), unless the build already points the tool elsewhere via its cache environment variable, in which case that path is cached as-is. This enables caching across CI steps in containerized environments.

Supported Tools and Configuration

Build Tool	Detection File	Config Method	Cache Location	Config Preserved
Maven	pom.xml	.mvn/maven.config	.m2/repository	✅ Appended
Gradle	build.gradle[.kts]	gradle.properties	.gradle	✅ Appended
Node.js	package-lock.json	existing npm_config_cache / .npmrc cache, or ~/.npm when that path is already inside the workspace	.npm	✅ Read-only. Does not write .npmrc
Yarn	yarn.lock	.yarnrc	.yarn/cache	✅ Appended
Go	go.mod	$GOPATH/pkg/mod	vendor (Go modules)	✅ Native
.NET	*.csproj/*.vbproj/.fsproj	Per-project dirs	bin/obj	✅ Native
Bazel	WORKSPACE/MODULE.bazel	.bazelrc	bazel-cache	✅ Appended
Poetry	poetry.lock	POETRY_CACHE_DIR / poetry.toml	.cache/poetry	✅ Merged
uv	uv.lock	UV_CACHE_DIR / uv.toml or pyproject.toml	.cache/uv	✅ Merged
Pipenv	Pipfile.lock	PIPENV_CACHE_DIR / .env	.cache/pipenv	✅ Merged
pip	requirements.txt / constraints.txt / pyproject.toml	PIP_CACHE_DIR / pip.conf	.cache/pip	✅ Merged
Fastlane	Gemfile (key adds Gemfile.lock)	.bundle/config BUNDLE_PATH	vendor/bundle	✅ Appended. An existing BUNDLE_PATH is kept
CocoaPods	Podfile (key adds Podfile.lock)	CP_CACHE_DIR / CP_HOME_DIR	Pods, plus ~/Library/Caches/CocoaPods and ~/.cocoapods/cache on macOS	✅ Read-only
Swift PM	Package.swift (key adds Package.resolved)	None	.build/checkouts, .build/repositories, plus ~/Library/Caches/org.swift.swiftpm on macOS	✅ Read-only

iOS Build Tool Details

The detected tool names are fastlane, cocoapods and spm. The cache key is the checksum of the manifest plus the checksum of its lockfile, so it changes when resolved versions change. Commit the lockfile: if the build rewrites it between restore and save (for example bundle update), the saved cache gets a different key and the next restore misses.

On Linux only the workspace directories are cached by default. CocoaPods download cache can be redirected to the workspace (e.g. /harness/.cocoapods-cache) on Linux or macOS by exporting CP_CACHE_DIR or CP_HOME_DIR. The shared $HOME caches (~/Library/Caches/CocoaPods and ~/.cocoapods/cache for CocoaPods; ~/Library/Caches/org.swift.swiftpm for SPM) are added on macOS, where the build and the cache step share the same HOME. Missing directories are skipped without failing the build. In Harness CI, enabling Cache Intelligence on the stage turns this on without extra steps.

steps:
  - name: restore-cache
    image: meltwater/drone-cache
    settings:
      restore: true
      auto_cache: true
  - name: build
    image: ruby:3.2
    commands:
      - bundle install # installs into vendor/bundle via .bundle/config
      - bundle exec fastlane ios build
  - name: rebuild-cache
    image: meltwater/drone-cache
    settings:
      rebuild: true
      auto_cache: true

Python Package Manager Details

Every Python tool resolves its cache location from an environment variable in preference to any config file. When the build already exports that variable the plugin caches exactly that path and writes no config file, because the tool would ignore the file anyway. The env var is therefore the reliable way to point a Python cache somewhere specific:

Tool	Environment variable
Poetry	POETRY_CACHE_DIR
uv	UV_CACHE_DIR
Pipenv	PIPENV_CACHE_DIR
pip	PIP_CACHE_DIR

The config-file fallbacks below apply only when the variable is unset.

Poetry: Modifies or creates Poetry's project-local poetry.toml file.

cache-dir = ".cache/poetry"

uv: Modifies uv.toml when it exists; otherwise it adds the setting to [tool.uv] in pyproject.toml. If neither file exists, it creates uv.toml. UV_CACHE_DIR outranks all three, so when it is set none of them are touched.

[tool.uv]
cache-dir = ".cache/uv"

Pipenv: Records the repo-local path in .env. Pipenv reads PIPENV_CACHE_DIR at process start and does not re-read .env afterwards, so later install steps must also have PIPENV_CACHE_DIR=.cache/pipenv exported for the .env fallback to have any effect.

PIPENV_CACHE_DIR=.cache/pipenv

pip: Writes pip.conf and caches .cache/pip. pip never loads a repository-local config file by itself, so for the fallback to work later install steps must set PIP_CONFIG_FILE=pip.conf or PIP_CACHE_DIR=.cache/pip. Detection files are requirements.txt, constraints.txt, and pyproject.toml (when no Poetry/uv/Pipenv lock is present).

[global]
cache-dir = .cache/pip

Only each tool's package-download cache is cached — not the .venv/venv directory itself. A venv bakes in absolute interpreter paths (e.g. symlinks into the base image's system Python), which can go dangling if a later step or build uses a different image, so it is rebuilt from the warm download cache on every run instead of being restored directly.

Example: Python Project with Poetry

For projects using Poetry (poetry.lock), the plugin will:

1. Automatically detect poetry.lock
2. Use POETRY_CACHE_DIR if the build exports it, otherwise modify poetry.toml to set Poetry cache to .cache/poetry
3. Cache and restore that directory across builds

No explicit mount configuration needed - just add the cache steps to your pipeline.

Example Usage of drone-cache

The following example configuration file (.drone.yml) shows the most common use of drone-cache.

Simple (with AWS S3 backend)

kind: pipeline
name: default

steps:
  - name: restore-cache
    image: meltwater/drone-cache
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: aws_access_key_id
      AWS_SECRET_ACCESS_KEY:
        from_secret: aws_secret_access_key
    pull: true
    settings:
      restore: true
      cache_key: {{ .Commit.Branch }}-{{ checksum "go.mod" }} # default if ommitted is {{ .Commit.Branch }}
      bucket: drone-cache-bucket
      region: eu-west-1
      mount:
        - 'vendor'

  - name: build
    image: golang:1.22.4-alpine3.19
    pull: true
    commands:
      - apk add --update make git
      - make drone-cache

  - name: rebuild-cache
    image: meltwater/drone-cache
    pull: true
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: aws_access_key_id
      AWS_SECRET_ACCESS_KEY:
        from_secret: aws_secret_access_key
    settings:
      rebuild: true
      cache_key: {{ .Commit.Branch }}-{{ checksum "go.mod" }} # default if ommitted is {{ .Commit.Branch }}
      bucket: drone-cache-bucket
      region: eu-west-1
      mount:
        - 'vendor'

More Examples

• examples for Drone, see docs/examples/drone-1.0.md

Usage

Using executable (with CLI args)

NAME:
   Drone cache plugin - Drone cache plugin

USAGE:
   drone-cache [global options] command [command options] [arguments...]

VERSION:
   v1.1.0

COMMANDS:
   help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --log.level value                     log filtering level. ('error', 'warn', 'info', 'debug') (default: "info") [$PLUGIN_LOG_LEVEL, $LOG_LEVEL]
   --log.format value                    log format to use. ('logfmt', 'json') (default: "logfmt") [$PLUGIN_LOG_FORMAT, $LOG_FORMAT]
   --repo.fullname value                 repository full name [$DRONE_REPO]
   --repo.namespace value                repository namespace [$DRONE_REPO_NAMESPACE]
   --repo.owner value                    repository owner (for Drone version < 1.0) [$DRONE_REPO_OWNER]
   --repo.name value                     repository name [$DRONE_REPO_NAME]
   --repo.link value                     repository link [$DRONE_REPO_LINK]
   --repo.avatar value                   repository avatar [$DRONE_REPO_AVATAR]
   --repo.branch value                   repository default branch [$DRONE_REPO_BRANCH]
   --repo.private                        repository is private (default: false) [$DRONE_REPO_PRIVATE]
   --repo.trusted                        repository is trusted (default: false) [$DRONE_REPO_TRUSTED]
   --remote.url value                    git remote url [$DRONE_REMOTE_URL]
   --commit.sha value                    git commit sha [$DRONE_COMMIT_SHA]
   --commit.ref value                    git commit ref (default: "refs/heads/master") [$DRONE_COMMIT_REF]
   --commit.branch value                 git commit branch (default: "master") [$DRONE_COMMIT_BRANCH]
   --commit.message value                git commit message [$DRONE_COMMIT_MESSAGE]
   --commit.link value                   git commit link [$DRONE_COMMIT_LINK]
   --commit.author.name value            git author name [$DRONE_COMMIT_AUTHOR]
   --commit.author.email value           git author email [$DRONE_COMMIT_AUTHOR_EMAIL]
   --commit.author.avatar value          git author avatar [$DRONE_COMMIT_AUTHOR_AVATAR]
   --build.event value                   build event (default: "push") [$DRONE_BUILD_EVENT]
   --build.number value                  build number (default: 0) [$DRONE_BUILD_NUMBER]
   --build.created value                 build created (default: 0) [$DRONE_BUILD_CREATED]
   --build.started value                 build started (default: 0) [$DRONE_BUILD_STARTED]
   --build.finished value                build finished (default: 0) [$DRONE_BUILD_FINISHED]
   --build.status value                  build status (default: "success") [$DRONE_BUILD_STATUS]
   --build.link value                    build link [$DRONE_BUILD_LINK]
   --build.deploy value                  build deployment target [$DRONE_DEPLOY_TO]
   --yaml.verified                       build yaml is verified (default: false) [$DRONE_YAML_VERIFIED]
   --yaml.signed                         build yaml is signed (default: false) [$DRONE_YAML_SIGNED]
   --prev.build.number value             previous build number (default: 0) [$DRONE_PREV_BUILD_NUMBER]
   --prev.build.status value             previous build status [$DRONE_PREV_BUILD_STATUS]
   --prev.commit.sha value               previous build sha [$DRONE_PREV_COMMIT_SHA]
   --backend value                       cache backend to use in plugin (s3, filesystem, sftp, azure, gcs) (default: "s3") [$PLUGIN_BACKEND]
   --mount value                         cache directories, an array of folders to cache [$PLUGIN_MOUNT]
   --rebuild                             rebuild the cache directories (default: false) [$PLUGIN_REBUILD]
   --restore                             restore the cache directories (default: false) [$PLUGIN_RESTORE]
   --cache-key value                     cache key to use for the cache directories [$PLUGIN_CACHE_KEY]
   --remote-root value                   remote root directory to contain all the cache files created (default repo.name) [$PLUGIN_REMOTE_ROOT]
   --local-root value                    local root directory to base given mount paths (default pwd [present working directory]) [$PLUGIN_LOCAL_ROOT]
   --override                            override even if cache key already exists in backend (default: true) [$PLUGIN_OVERRIDE]
   --archive-format value                archive format to use to store the cache directories (tar, gzip, zstd) (default: "tar") [$PLUGIN_ARCHIVE_FORMAT]
   --compression-level value             compression level to use for gzip/zstd compression when archive-format specified as gzip/zstd
                                             (check https://godoc.org/compress/flate#pkg-constants for available options for gzip
                                             and https://pkg.go.dev/github.com/klauspost/compress/zstd#EncoderLevelFromZstd for zstd) (default: -1)
   --skip-symlinks                       skip symbolic links in archive (default: false) [$PLUGIN_SKIP_SYMLINKS, $SKIP_SYMLINKS]
   --preserve-metadata                   preserve file metadata (mode, ownership, timestamps) (default: false) [$PLUGIN_PRESERVE_METADATA]
   --debug                               debug (default: false) [$PLUGIN_DEBUG, $DEBUG]
   --backend.operation-timeout value     timeout value to use for each storage operations (default: 3m0s) [$PLUGIN_BACKEND_OPERATION_TIMEOUT, $BACKEND_OPERATION_TIMEOUT]
   --endpoint value                      endpoint for the s3/cloud storage connection [$PLUGIN_ENDPOINT, $S3_ENDPOINT, $GCS_ENDPOINT]
   --bucket value                        AWS bucket name [$PLUGIN_BUCKET, $S3_BUCKET, $GCS_BUCKET]
   --filesystem.cache-root value         local filesystem root directory for the filesystem cache (default: "/tmp/cache") [$PLUGIN_FILESYSTEM_CACHE_ROOT, $FILESYSTEM_CACHE_ROOT]
   --access-key value                    AWS access key [$PLUGIN_ACCESS_KEY, $AWS_ACCESS_KEY_ID, $CACHE_AWS_ACCESS_KEY_ID]
   --secret-key value                    AWS secret key [$PLUGIN_SECRET_KEY, $AWS_SECRET_ACCESS_KEY, $CACHE_AWS_SECRET_ACCESS_KEY]
   --region value                        AWS bucket region. (us-east-1, eu-west-1, ...) [$PLUGIN_REGION, $S3_REGION]
   --path-style                          AWS path style to use for bucket paths. (true for minio, false for aws) (default: false) [$PLUGIN_PATH_STYLE, $AWS_PLUGIN_PATH_STYLE]
   --acl value                           upload files with acl (private, public-read, ...) [$PLUGIN_ACL, $AWS_ACL]
   --encryption value                    server-side encryption algorithm, defaults to none. (AES256, aws:kms) [$PLUGIN_ENCRYPTION, $AWS_ENCRYPTION]
   --sts-endpoint value                  Custom STS endpoint for IAM role assumption [$PLUGIN_STS_ENDPOINT, $AWS_STS_ENDPOINT]
   --role-arn value                      AWS IAM role ARN to assume [$PLUGIN_ASSUME_ROLE_ARN, $AWS_ASSUME_ROLE_ARN]
   --gcs.api-key value                   Google service account API key [$PLUGIN_API_KEY, $GCP_API_KEY]
   --gcs.json-key value                  Google service account JSON key [$PLUGIN_JSON_KEY, $GCS_CACHE_JSON_KEY]
   --gcs.acl value                       upload files with acl (private, public-read, ...) (default: "private") [$PLUGIN_GCS_ACL, $GCS_ACL]
   --gcs.encryption-key value            server-side encryption key, must be a 32-byte AES-256 key, defaults to none
                                             (See https://cloud.google.com/storage/docs/encryption for details.) [$PLUGIN_GCS_ENCRYPTION_KEY, $GCS_ENCRYPTION_KEY]
   --azure.account-name value            Azure Blob Storage Account Name [$PLUGIN_ACCOUNT_NAME, $AZURE_ACCOUNT_NAME]
   --azure.account-key value             Azure Blob Storage Account Key [$PLUGIN_ACCOUNT_KEY, $AZURE_ACCOUNT_KEY]
   --azure.container-name value          Azure Blob Storage container name [$PLUGIN_CONTAINER, $AZURE_CONTAINER_NAME]
   --azure.blob-storage-url value        Azure Blob Storage URL (default: "blob.core.windows.net") [$AZURE_BLOB_STORAGE_URL]
   --azure.blob-max-retry-requets value  Azure Blob Storage Max Retry Requests (default: 4) [$AZURE_BLOB_MAX_RETRY_REQUESTS]
   --sftp.cache-root value               sftp root directory [$SFTP_CACHE_ROOT]
   --sftp.username value                 sftp username [$PLUGIN_USERNAME, $SFTP_USERNAME]
   --sftp.password value                 sftp password [$PLUGIN_PASSWORD, $SFTP_PASSWORD]
   --sftp.public-key-file value          sftp public key file path [$PLUGIN_PUBLIC_KEY_FILE, $SFTP_PUBLIC_KEY_FILE]
   --sftp.auth-method value              sftp auth method, defaults to none. (PASSWORD, PUBLIC_KEY_FILE) [$SFTP_AUTH_METHOD]
   --sftp.host value                     sftp host [$SFTP_HOST]
   --sftp.port value                     sftp port [$SFTP_PORT]
   --help, -h                            show help (default: false)
   --version, -v                         print the version (default: false)

Using Docker (with Environment variables)

$ docker run --rm \
      -v "$(pwd)":/app \
      -e DRONE_REPO=octocat/hello-world \
      -e DRONE_REPO_BRANCH=master \
      -e DRONE_COMMIT_BRANCH=master \
      -e PLUGIN_MOUNT=/app/node_modules \
      -e PLUGIN_RESTORE=false \
      -e PLUGIN_REBUILD=true \
      -e PLUGIN_BUCKET=<bucket> \
      -e AWS_ACCESS_KEY_ID=<token> \
      -e AWS_SECRET_ACCESS_KEY=<secret> \
      meltwater/drone-cache

Development

Usage:
  make <target>

Targets:
  setup          	  Setups dev environment
  drone-cache    	  Runs drone-cache target
  clean          	  Cleans build resourcess
  docs           	  Generates docs
  generate       	  Generate documentation, website and yaml files,
  vendor         	  Updates vendored copy of dependencies
  compress       	  Creates compressed binary
  container      	  Builds drone-cache docker image with latest tag
  container-push 	  Pushes latest $(CONTAINER_REPO) image to repository
  test           	  Runs tests
  test-integration	  Runs integration tests
  test-unit      	  Runs unit tests
  lint           	  Runs golangci-lint analysis
  fix            	  Runs golangci-lint fix
  format         	  Runs gofmt
  help           	  Shows this help message

Configuration Variables for Secondary Role Assumption with External ID

The following environment variables enable the plugin to assume a secondary IAM role using IRSA, with an External ID if required by the role’s trust policy.

Variables

PLUGIN_USER_ROLE_ARN

• Type: String
• Required: No
• Description: Specifies the secondary IAM role to be assumed by the plugin, allowing it to inherit permissions associated with this role and access specific AWS resources.

PLUGIN_USER_ROLE_EXTERNAL_ID

• Type: String
• Required: No
• Description: Provide the External ID necessary for the role assumption process if the secondary role’s trust policy mandates it. This is often required for added security, ensuring that only authorized entities assume the role.

Usage Notes

• If the role secondary role (PLUGIN_USER_ROLE_ARN) requires an External ID then pass it through PLUGIN_USER_ROLE_EXTERNAL_ID.

Releases

Release management handled by the CI pipeline. When you create a tag on master branch, CI handles the rest.

You can find released artifacts (binaries, code, archives) under releases.

You can find released images at DockerHub.

PLEASE DO NOT INTRODUCE BREAKING CHANGES

Keep in mind that users usually use the image tagged with latest in their pipeline, please make sure you do not interfere with their working workflow. Latest stable releases will be tagged with the latest.

Versioning

drone-cache uses SemVer for versioning. For the versions available, see the tags on this repository.

As the versioning scheme dictates, drone-cache respects backward compatibility within the major versions. However, the project only offers guarantees regarding the command-line interface (flags and environment variables). Any exported public package can change its API.

Authors and Acknowledgement

See the list of all contributors.

• @dim - Thanks for original work!
• @AdamGlazerMW - Special thanks to Adam for the amazing artwork!

Inspiration

• github.com/bsm/drone-s3-cache (original work)
• github.com/Drillster/drone-volume-cache
• github.com/drone/drone-cache-lib

Contributing

Please read CONTRIBUTING.md to understand how to submit pull requests to us, and also see our code of conduct.

Future work

All ideas for new features and bug reports will be kept in github.com/meltwater/drone-cache/issues.

One bigger area of future investment is to add a couple of new storage backends for caching the workspace files.

License and Copyright

This project is licensed under the Apache License 2.0. .