Plugins

Every step is a plugin's. lely steps lists the ones this project can use, with their options, what they give, and what they can do.

bundle

A Databricks Asset Bundle: planned, deployed, listed and destroyed through the Databricks CLI. lely never reimplements what the CLI resolves — it asks.

  - name: app
    uses: bundle
    with:
      path: .                    # where databricks.yml is; the default
      vars:
        model_version: ${steps.model.version}

Option
path	The bundle's folder, from the config's. Default .
vars	Bundle variables, passed as --var. A value may come from a step above.

It gives the bundle's target and name, workspace.<field>, var.<name>, every resource's own config as resources.<type>.<key>.<field> — known at plan — and resources.<type>.<key>.id and .url, known once the resource exists.

Every bundle plan carries one run: uploads the bundle's files. A deploy uploads them even when no resource changes, so a bundle step is never "nothing to do".

The CLI trusts the bundle with your credentials

A bundle whose target names another host is not refused by the CLI: with a token from the environment it goes to that host and presents the token. lely compares the two hosts and stops, but only after the CLI's first call. Know whose databricks.yml you run.

command

Commands you give it, for steps that don't need Python.

  - name: seed
    uses: command
    with:
      plan: [./ops/seed.sh, --plan]      # optional
      apply: [./ops/seed.sh]
      destroy: [./ops/seed.sh, --drop]   # optional
      outputs: [count]                   # optional: what the step gives

Each command is a list — a program and its arguments — and is never passed through a shell. All run in the project's directory, with the environment lely was run in, the step's env, and LELY_TARGET and LELY_STEP.

• plan prints the step's plan as JSON on stdout:

  {"changes": [{"key": "users", "action": "create", "summary": "seed 3 users"}],
   "outputs": {"count": 3}}
  

  Without a plan command the step's plan is a single run: it runs on every apply. lely can't show what a script it has never run will do.

• apply also gets LELY_PLAN, a file holding the plan that was approved for the step, and LELY_OUTPUTS, a file it writes its outputs to, one name=value to a line.

• destroy is shown in the destroy plan in full, marked destructive: lely can't vouch for what it removes. Without one the step is skipped, visibly.
• outputs lists what the step gives. One the plan command prints is known at plan; one the apply command writes is known only after the run.
• env is extra environment for every command, and may hold a secret. Arguments may not.

bundle.run

Runs a job, pipeline or app from a bundle step, with databricks bundle run.

  - name: backfill
    uses: bundle.run
    with: {bundle: app, resource: jobs.backfill, args: ["--full"]}

It stands below the bundle step it names, and its plan is a run.

Not run for real yet

bundle.run is tested against the fake CLI only: no job has been started with it on a workspace.

Your own

A class in a file in your repo, or one a package registers:

  - name: model
    uses: ./ops/steps.py:LatestModel       # a class in the repo
  - name: audit
    uses: acme_deploy.steps:Audit          # a class in an installed package

→ Writing a plugin.

Planning runs your project's own code — a plugin in the repo, a command step's plan command. On a pull request, give lely plan credentials that can read and nothing more.

stevin

Tables, planned by stevin. Parked: it can plan, and can't apply yet — lely apply refuses a project that uses it.