Compare commits

..

44 Commits

Author SHA1 Message Date
Andrey Nering
21531b6291 v3.37.1 2024-05-09 11:22:47 -03:00
Andrey Nering
bfc9d7847d fix: add changelog + fix for booleans for #1641 2024-05-09 11:21:12 -03:00
Valentin Maerten
3397f2855f fix: handle int and float env variable by converting them to string (#1641) 2024-05-09 11:14:38 -03:00
Jordan
78a69c4c3e chore: fix json schema typos (#1642) 2024-05-09 14:11:39 +00:00
Pete Davison
01716f55b3 chore: prep any variables for release (#1586)
* chore: release blog post

* chore: rename blog post to any-variables

* chore: update the release version in the blog

* chore: update blog date
2024-05-09 10:17:03 +01:00
Andrey Nering
ca364c20bb chore(goreleaser): fix deprecation warning 2024-05-08 21:40:50 -03:00
Andrey Nering
ee901fe568 v3.37.0 2024-05-08 21:32:16 -03:00
Pete Davison
7fa06eedf4 chore: changelog and docs for #1623 2024-05-08 15:49:01 +00:00
Pete Davison
651033c5a7 feat: stdin required -t - (#1623) 2024-05-08 16:44:05 +01:00
Valentin Maerten
17f6e816d8 fix(remote): do not display prompt if it's empty (#1634) 2024-05-05 16:10:32 +01:00
Pete Davison
cd259a741f chore: changelog for #1610 2024-04-29 21:32:42 +00:00
Valentin Maerten
c81dbda157 feat(remote): replace env variable in include remote URL (#1610)
* feat(remote): replace env variable in include remote URL

* use templating system instead of os.ExpandEnv

* lint
2024-04-29 22:27:30 +01:00
Michael Zhao
e23ef818ea docs: fix reference to GOOS and GOARCH link (#1628) 2024-04-29 15:01:18 -03:00
Pete Davison
ddd9964db7 feat: warn about move from any variables to map variables (#1618) 2024-04-24 21:40:52 +01:00
Pete Davison
a5b949f5dc chore: changelog for #1612 2024-04-24 19:50:03 +00:00
Pete Davison
630e58767b feat: ability to resolve refs using templating syntax (#1612)
* feat: resolve references using templating syntax

* refactor: moved when references are resolved to one place

* fix: linter

* docs: update map variables doc
2024-04-24 19:47:24 +00:00
Pete Davison
d87e5de56f chore: changelog for #1607 2024-04-24 17:35:48 +00:00
Pete Davison
f75aa1f84b feat: taskfile mutex for adding edge data 2024-04-24 18:33:56 +01:00
Pete Davison
53235f07ad feat: edge weight 2024-04-24 18:33:56 +01:00
Pete Davison
f19c520f23 feat: add support for multiple includes on a graph edge 2024-04-24 18:33:56 +01:00
Pete Davison
6951e5cd0c refactor: includes uses pointers 2024-04-24 18:33:56 +01:00
Andrey Nering
24059a4b76 chore(changelog): add entry for #1613 2024-04-23 22:58:56 -03:00
jwater7
fa022be1f9 chore(completions): support tilde home directory for zsh (#1613) 2024-04-24 01:57:43 +00:00
Andrey Nering
a3b9554efd chore: improve changelog for #1603 2024-04-23 22:49:12 -03:00
Tim Vergenz
16070c7a24 feat: add alias q for template function shellQuote (#1603)
Resolves #1601
2024-04-23 22:47:40 -03:00
Andrey Nering
72d9671fcf chore(website): disable translations for now (#1617) 2024-04-24 01:23:06 +00:00
Pete Davison
d01b3c8979 chore: changelog for #1563 2024-04-09 11:41:28 +00:00
Pete Davison
4024b4fa37 chore: remove code that outputs the graphviz file 2024-04-09 12:37:18 +01:00
Pete Davison
54c7f35b00 fix: linting issues 2024-04-09 12:37:18 +01:00
Pete Davison
3efb437c9a feat: merge concurrency 2024-04-09 12:37:18 +01:00
Pete Davison
e9448bd4be fix: advanced import operates on including file instead of included file 2024-04-09 12:37:18 +01:00
Pete Davison
8f3180a9fa fix: bug with merge code 2024-04-09 12:37:18 +01:00
Pete Davison
1d230af90d fix: advanced import resolving dynamic variables incorrectly 2024-04-09 12:37:18 +01:00
Pete Davison
fb9f6c20ab feat: merger 2024-04-09 12:37:18 +01:00
Pete Davison
6854b4c300 fix: include_with_vars test included the same file multiple times 2024-04-09 12:37:18 +01:00
Pete Davison
b10c573270 fix: missing task locations 2024-04-09 12:37:18 +01:00
Pete Davison
6ecfb634d2 fix: includes interpolation test 2024-04-09 12:37:18 +01:00
Pete Davison
6b3f8e29bb fix: optional includes 2024-04-09 12:37:18 +01:00
Pete Davison
220bf74a9e feat: better taskfile cycle error handling 2024-04-09 12:37:18 +01:00
Pete Davison
0a027df50d feat: better error handling for duplicate edges and fixed tests 2024-04-09 12:37:18 +01:00
Pete Davison
a50580b5a1 feat: dag reader 2024-04-09 12:37:18 +01:00
Pete Davison
1890722b75 chore: changelog for #1547 2024-04-09 11:28:12 +00:00
Pete Davison
1ff618cc17 feat: enable any variables without maps (#1547)
* feat: enable any variable experiment (without maps)

* chore: rename any_variables experiment to map_variables

* docs: create map variables experiment docs and update usage

* blog: any variables

* fix: links

* fix: warn about broken links instead of failing
2024-04-09 12:14:14 +01:00
Andrey Nering
eb2783fcce fix: fix bug for files with special chars &() (#1584) 2024-04-09 02:08:30 +00:00
64 changed files with 1120 additions and 567 deletions

3
.gitignore vendored
View File

@@ -10,6 +10,9 @@
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Graphvis files
*.gv
# Project-local glide cache, RE: https://github.com/Masterminds/glide/issues/736
.glide/

View File

@@ -11,7 +11,7 @@ linters:
linters-settings:
goimports:
local-prefixes: github.com/go-task/task
local-prefixes: github.com/go-task
gofmt:
rewrite-rules:
- pattern: 'interface{}'

View File

@@ -71,7 +71,7 @@ brews:
description: Task runner / simpler Make alternative written in Go
license: MIT
homepage: https://taskfile.dev
folder: Formula
directory: Formula
repository:
owner: go-task
name: homebrew-tap

View File

@@ -1,5 +1,31 @@
# Changelog
## v3.37.1 - 2024-05-09
- Fix bug where non-string values (numbers, bools) added to `env:` weren't been
correctly exported (#1640, #1641 by @vmaerten and @andreynering).
## v3.37.0 - 2024-05-08
- Released the
[Any Variables experiment](https://taskfile.dev/blog/any-variables), but
[_without support for maps_](https://github.com/go-task/task/issues/1415#issuecomment-2044756925)
(#1415, #1547 by @pd93).
- Refactored how Task reads, parses and merges Taskfiles using a DAG (#1563,
#1607 by @pd93).
- Fix a bug which stopped tasks from using `stdin` as input (#1593, #1623 by
@pd03).
- Fix error when a file or directory in the project contained a special char
like `&`, `(` or `)` (#1551, #1584 by @andreynering).
- Added alias `q` for template function `shellQuote` (#1601, #1603 by @vergenzt)
- Added support for `~` on ZSH completions (#1613 by @jwater7).
- Added the ability to pass variables by reference using Go template syntax when
the
[Map Variables experiment](https://taskfile.dev/experiments/map-variables/) is
enabled (#1612 by @pd93).
- Added support for environment variables in the templating engine in `includes`
(#1610 by @vmaerten).
## v3.36.0 - 2024-04-08
- Added support for

View File

@@ -3,7 +3,6 @@ package main
import (
"context"
"fmt"
"log"
"os"
"strings"
@@ -44,8 +43,12 @@ func main() {
}
func run() error {
log.SetFlags(0)
log.SetOutput(os.Stderr)
logger := &logger.Logger{
Stdout: os.Stdout,
Stderr: os.Stderr,
Verbose: flags.Verbose,
Color: flags.Color,
}
if err := flags.Validate(); err != nil {
return err
@@ -65,22 +68,16 @@ func run() error {
}
if flags.Experiments {
l := &logger.Logger{
Stdout: os.Stdout,
Stderr: os.Stderr,
Verbose: flags.Verbose,
Color: flags.Color,
}
return experiments.List(l)
return experiments.List(logger)
}
if flags.Init {
wd, err := os.Getwd()
if err != nil {
log.Fatal(err)
return err
}
if err := task.InitTaskfile(os.Stdout, wd); err != nil {
log.Fatal(err)
return err
}
return nil
}
@@ -138,6 +135,10 @@ func run() error {
return err
}
if experiments.AnyVariables.Enabled {
logger.Warnf("The 'Any Variables' experiment flag is no longer required to use non-map variable types. If you wish to use map variables, please use 'TASK_X_MAP_VARIABLES' instead. See https://github.com/go-task/task/issues/1585\n")
}
// If the download flag is specified, we should stop execution as soon as
// taskfile is downloaded
if flags.Download {
@@ -178,7 +179,7 @@ func run() error {
globals.Set("CLI_ARGS", ast.Var{Value: cliArgs})
globals.Set("CLI_FORCE", ast.Var{Value: flags.Force || flags.ForceAll})
e.Taskfile.Vars.Merge(globals)
e.Taskfile.Vars.Merge(globals, nil)
if !flags.Watch {
e.InterceptInterruptSignals()

View File

@@ -12,7 +12,9 @@ function __task_list() {
local taskfile item task desc
cmd=(task)
taskfile="${(v)opt_args[(i)-t|--taskfile]}"
taskfile=${(Qv)opt_args[(i)-t|--taskfile]}
taskfile=${taskfile//\~/$HOME}
if [[ -n "$taskfile" && -f "$taskfile" ]]; then
enabled=1

View File

@@ -19,6 +19,8 @@ const (
CodeTaskfileCacheNotFound
CodeTaskfileVersionCheckError
CodeTaskfileNetworkTimeout
_ // CodeTaskfileDuplicateInclude
CodeTaskfileCycle
)
// Task related exit codes

View File

@@ -174,3 +174,21 @@ func (err *TaskfileNetworkTimeoutError) Error() string {
func (err *TaskfileNetworkTimeoutError) Code() int {
return CodeTaskfileNetworkTimeout
}
// TaskfileCycleError is returned when we detect that a Taskfile includes a
// set of Taskfiles that include each other in a cycle.
type TaskfileCycleError struct {
Source string
Destination string
}
func (err TaskfileCycleError) Error() string {
return fmt.Sprintf("task: include cycle detected between %s <--> %s",
err.Source,
err.Destination,
)
}
func (err TaskfileCycleError) Code() int {
return CodeTaskfileCycle
}

4
go.mod
View File

@@ -1,12 +1,14 @@
module github.com/go-task/task/v3
go 1.21
go 1.21.0
require (
github.com/Masterminds/semver/v3 v3.2.1
github.com/davecgh/go-spew v1.1.1
github.com/dominikbraun/graph v0.23.0
github.com/fatih/color v1.16.0
github.com/go-task/slim-sprig/v3 v3.0.0
github.com/go-task/template v0.0.0-20240422130016-8f6b279b1e90
github.com/joho/godotenv v1.5.1
github.com/mattn/go-zglob v0.0.4
github.com/mitchellh/hashstructure/v2 v2.0.2

4
go.sum
View File

@@ -4,12 +4,16 @@ github.com/creack/pty v1.1.21 h1:1/QdRyBaHHJP61QkWMXlOIBfsgdDeeKfK8SYVUWJKf0=
github.com/creack/pty v1.1.21/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dominikbraun/graph v0.23.0 h1:TdZB4pPqCLFxYhdyMFb1TBdFxp8XLcJfTTBQucVPgCo=
github.com/dominikbraun/graph v0.23.0/go.mod h1:yOjYyogZLY1LSG9E33JWZJiq5k83Qy2C6POAuiViluc=
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
github.com/go-task/template v0.0.0-20240422130016-8f6b279b1e90 h1:JBbiZ2CXIZ9Upe3O2yI5+3ksWoa7hNVNi4BINs8TIrs=
github.com/go-task/template v0.0.0-20240422130016-8f6b279b1e90/go.mod h1:RgwRaZK+kni/hJJ7/AaOE2lPQFPbAdji/DyhC6pxo4k=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=

View File

@@ -62,10 +62,6 @@ func (c *Compiler) getVariables(t *ast.Task, call *ast.Call, evaluateShVars bool
cache := &templater.Cache{Vars: result}
// Replace values
newVar := templater.ReplaceVar(v, cache)
// If the variable is a reference, we can resolve it
if newVar.Ref != "" {
newVar.Value = result.Get(newVar.Ref).Value
}
// If the variable should not be evaluated, but is nil, set it to an empty string
// This stops empty interface errors when using the templater to replace values later
if !evaluateShVars && newVar.Value == nil {

16
internal/env/env.go vendored
View File

@@ -13,19 +13,25 @@ func Get(t *ast.Task) []string {
}
environ := os.Environ()
for k, v := range t.Env.ToCacheMap() {
str, isString := v.(string)
if !isString {
if !isTypeAllowed(v) {
continue
}
if _, alreadySet := os.LookupEnv(k); alreadySet {
continue
}
environ = append(environ, fmt.Sprintf("%s=%s", k, str))
environ = append(environ, fmt.Sprintf("%s=%v", k, v))
}
return environ
}
func isTypeAllowed(v any) bool {
switch v.(type) {
case string, bool, int, float32, float64:
return true
default:
return false
}
}

View File

@@ -103,6 +103,9 @@ func IsExitError(err error) bool {
func Expand(s string) (string, error) {
s = filepath.ToSlash(s)
s = strings.ReplaceAll(s, " ", `\ `)
s = strings.ReplaceAll(s, "&", `\&`)
s = strings.ReplaceAll(s, "(", `\(`)
s = strings.ReplaceAll(s, ")", `\)`)
fields, err := shell.Fields(s, nil)
if err != nil {
return "", err

View File

@@ -28,6 +28,7 @@ var (
GentleForce Experiment
RemoteTaskfiles Experiment
AnyVariables Experiment
MapVariables Experiment
)
func init() {
@@ -35,6 +36,7 @@ func init() {
GentleForce = New("GENTLE_FORCE")
RemoteTaskfiles = New("REMOTE_TASKFILES")
AnyVariables = New("ANY_VARIABLES", "1", "2")
MapVariables = New("MAP_VARIABLES", "1", "2")
}
func New(xName string, enabledValues ...string) Experiment {
@@ -101,6 +103,6 @@ func List(l *logger.Logger) error {
w := tabwriter.NewWriter(os.Stdout, 0, 8, 0, ' ', 0)
printExperiment(w, l, GentleForce)
printExperiment(w, l, RemoteTaskfiles)
printExperiment(w, l, AnyVariables)
printExperiment(w, l, MapVariables)
return w.Flush()
}

View File

@@ -3,6 +3,7 @@ package flags
import (
"errors"
"log"
"os"
"time"
"github.com/spf13/pflag"
@@ -68,6 +69,8 @@ var (
)
func init() {
log.SetFlags(0)
log.SetOutput(os.Stderr)
pflag.Usage = func() {
log.Print(usage)
pflag.PrintDefaults()

View File

@@ -138,6 +138,10 @@ func (l *Logger) VerboseErrf(color Color, s string, args ...any) {
}
}
func (l *Logger) Warnf(message string, args ...any) {
l.Errf(Yellow, message, args...)
}
func (l *Logger) Prompt(color Color, prompt string, defaultValue string, continueValues ...string) error {
if l.AssumeYes {
l.Outf(color, "%s [assuming yes]\n", prompt)

View File

@@ -4,13 +4,13 @@ import (
"path/filepath"
"runtime"
"strings"
"text/template"
"github.com/davecgh/go-spew/spew"
"mvdan.cc/sh/v3/shell"
"mvdan.cc/sh/v3/syntax"
sprig "github.com/go-task/slim-sprig/v3"
"github.com/go-task/template"
)
var templateFuncs template.FuncMap
@@ -73,12 +73,16 @@ func init() {
return spew.Sdump(v)
},
}
// aliases
taskFuncs["q"] = taskFuncs["shellQuote"]
// Deprecated aliases for renamed functions.
taskFuncs["FromSlash"] = taskFuncs["fromSlash"]
taskFuncs["ToSlash"] = taskFuncs["toSlash"]
taskFuncs["ExeExt"] = taskFuncs["exeExt"]
templateFuncs = sprig.TxtFuncMap()
templateFuncs = template.FuncMap(sprig.TxtFuncMap())
for k, v := range taskFuncs {
templateFuncs[k] = v
}

View File

@@ -4,10 +4,10 @@ import (
"bytes"
"maps"
"strings"
"text/template"
"github.com/go-task/task/v3/internal/deepcopy"
"github.com/go-task/task/v3/taskfile/ast"
"github.com/go-task/template"
)
// Cache is a help struct that allow us to call "replaceX" funcs multiple
@@ -29,6 +29,25 @@ func (r *Cache) Err() error {
return r.err
}
func ResolveRef(ref string, cache *Cache) any {
// If there is already an error, do nothing
if cache.err != nil {
return nil
}
// Initialize the cache map if it's not already initialized
if cache.cacheMap == nil {
cache.cacheMap = cache.Vars.ToCacheMap()
}
val, err := template.ResolveRef(ref, cache.cacheMap)
if err != nil {
cache.err = err
return nil
}
return val
}
func Replace[T any](v T, cache *Cache) T {
return ReplaceWithExtra(v, cache, nil)
}
@@ -91,6 +110,9 @@ func ReplaceVar(v ast.Var, cache *Cache) ast.Var {
}
func ReplaceVarWithExtra(v ast.Var, cache *Cache, extra map[string]any) ast.Var {
if v.Ref != "" {
return ast.Var{Value: ResolveRef(v.Ref, cache)}
}
return ast.Var{
Value: ReplaceWithExtra(v.Value, cache, extra),
Sh: ReplaceWithExtra(v.Sh, cache, extra),

2
package-lock.json generated
View File

@@ -1,6 +1,6 @@
{
"name": "@go-task/cli",
"version": "3.36.0",
"version": "3.37.1",
"lockfileVersion": 2,
"requires": true,
"packages": {

View File

@@ -1,6 +1,6 @@
{
"name": "@go-task/cli",
"version": "3.36.0",
"version": "3.37.1",
"description": "A task runner / simpler Make alternative written in Go",
"scripts": {
"postinstall": "go-npm install",

View File

@@ -63,8 +63,7 @@ func (e *Executor) getRootNode() (taskfile.Node, error) {
}
func (e *Executor) readTaskfile(node taskfile.Node) error {
var err error
e.Taskfile, err = taskfile.Read(
reader := taskfile.NewReader(
node,
e.Insecure,
e.Download,
@@ -73,9 +72,13 @@ func (e *Executor) readTaskfile(node taskfile.Node) error {
e.TempDir,
e.Logger,
)
graph, err := reader.Read()
if err != nil {
return err
}
if e.Taskfile, err = graph.Merge(); err != nil {
return err
}
return nil
}

View File

@@ -101,8 +101,9 @@ func TestEnv(t *testing.T) {
Target: "default",
TrimSpace: false,
Files: map[string]string{
"local.txt": "GOOS='linux' GOARCH='amd64' CGO_ENABLED='0'\n",
"global.txt": "FOO='foo' BAR='overriden' BAZ='baz'\n",
"local.txt": "GOOS='linux' GOARCH='amd64' CGO_ENABLED='0'\n",
"global.txt": "FOO='foo' BAR='overriden' BAZ='baz'\n",
"multiple_type.txt": "FOO='1' BAR='true' BAZ='1.1'\n",
},
}
tt.Run(t)
@@ -1199,15 +1200,17 @@ func TestIncludesInterpolation(t *testing.T) {
expectedErr bool
expectedOutput string
}{
{"include", "include", false, "includes_interpolation\n"},
{"include with dir", "include-with-dir", false, "included\n"},
{"include", "include", false, "include\n"},
{"include_with_env_variable", "include-with-env-variable", false, "include_with_env_variable\n"},
{"include_with_dir", "include-with-dir", false, "included\n"},
}
t.Setenv("MODULE", "included")
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
var buff bytes.Buffer
e := task.Executor{
Dir: dir,
Dir: filepath.Join(dir, test.name),
Stdout: &buff,
Stderr: &buff,
Silent: true,

129
taskfile/ast/graph.go Normal file
View File

@@ -0,0 +1,129 @@
package ast
import (
"fmt"
"os"
"sync"
"github.com/dominikbraun/graph"
"github.com/dominikbraun/graph/draw"
"golang.org/x/sync/errgroup"
)
type TaskfileGraph struct {
sync.Mutex
graph.Graph[string, *TaskfileVertex]
}
// A TaskfileVertex is a vertex on the Taskfile DAG.
type TaskfileVertex struct {
URI string
Taskfile *Taskfile
}
func taskfileHash(vertex *TaskfileVertex) string {
return vertex.URI
}
func NewTaskfileGraph() *TaskfileGraph {
return &TaskfileGraph{
sync.Mutex{},
graph.New(taskfileHash,
graph.Directed(),
graph.PreventCycles(),
graph.Rooted(),
),
}
}
func (tfg *TaskfileGraph) Visualize(filename string) error {
f, err := os.Create(filename)
if err != nil {
return err
}
defer f.Close()
return draw.DOT(tfg.Graph, f)
}
func (tfg *TaskfileGraph) Merge() (*Taskfile, error) {
hashes, err := graph.TopologicalSort(tfg.Graph)
if err != nil {
return nil, err
}
predecessorMap, err := tfg.PredecessorMap()
if err != nil {
return nil, err
}
// Loop over each vertex in reverse topological order except for the root vertex.
// This gives us a loop over every included Taskfile in an order which is safe to merge.
for i := len(hashes) - 1; i > 0; i-- {
hash := hashes[i]
// Get the included vertex
includedVertex, err := tfg.Vertex(hash)
if err != nil {
return nil, err
}
// Create an error group to wait for all the included Taskfiles to be merged with all its parents
var g errgroup.Group
// Loop over edge that leads to a vertex that includes the current vertex
for _, edge := range predecessorMap[hash] {
// Start a goroutine to process each included Taskfile
g.Go(func() error {
// Get the base vertex
vertex, err := tfg.Vertex(edge.Source)
if err != nil {
return err
}
// Get the merge options
includes, ok := edge.Properties.Data.([]*Include)
if !ok {
return fmt.Errorf("task: Failed to get merge options")
}
// Merge the included Taskfiles into the parent Taskfile
for _, include := range includes {
if err := vertex.Taskfile.Merge(
includedVertex.Taskfile,
include,
); err != nil {
return err
}
}
return nil
})
if err := g.Wait(); err != nil {
return nil, err
}
}
// Wait for all the go routines to finish
if err := g.Wait(); err != nil {
return nil, err
}
}
// Get the root vertex
rootVertex, err := tfg.Vertex(hashes[0])
if err != nil {
return nil, err
}
_ = rootVertex.Taskfile.Tasks.Range(func(name string, task *Task) error {
if task == nil {
task = &Task{}
rootVertex.Taskfile.Tasks.Set(name, task)
}
task.Task = name
return nil
})
return rootVertex.Taskfile, nil
}

View File

@@ -22,7 +22,7 @@ type Include struct {
// Includes represents information about included tasksfiles
type Includes struct {
omap.OrderedMap[string, Include]
omap.OrderedMap[string, *Include]
}
// UnmarshalYAML implements the yaml.Unmarshaler interface.
@@ -41,7 +41,7 @@ func (includes *Includes) UnmarshalYAML(node *yaml.Node) error {
return err
}
v.Namespace = keyNode.Value
includes.Set(keyNode.Value, v)
includes.Set(keyNode.Value, &v)
}
return nil
}
@@ -58,7 +58,7 @@ func (includes *Includes) Len() int {
}
// Wrapper around OrderedMap.Set to ensure we don't get nil pointer errors
func (includes *Includes) Range(f func(k string, v Include) error) error {
func (includes *Includes) Range(f func(k string, v *Include) error) error {
if includes == nil {
return nil
}

View File

@@ -1,6 +1,7 @@
package ast
import (
"errors"
"fmt"
"time"
@@ -13,6 +14,9 @@ const NamespaceSeparator = ":"
var V3 = semver.MustParse("3")
// ErrIncludedTaskfilesCantHaveDotenvs is returned when a included Taskfile contains dotenvs
var ErrIncludedTaskfilesCantHaveDotenvs = errors.New("task: Included Taskfiles can't have dotenv declarations. Please, move the dotenv declaration to the main Taskfile")
// Taskfile is the abstract syntax tree for a Taskfile
type Taskfile struct {
Location string
@@ -36,6 +40,9 @@ func (t1 *Taskfile) Merge(t2 *Taskfile, include *Include) error {
if !t1.Version.Equal(t2.Version) {
return fmt.Errorf(`task: Taskfiles versions should match. First is "%s" but second is "%s"`, t1.Version, t2.Version)
}
if len(t2.Dotenv) > 0 {
return ErrIncludedTaskfilesCantHaveDotenvs
}
if t2.Output.IsSet() {
t1.Output = t2.Output
}
@@ -45,9 +52,9 @@ func (t1 *Taskfile) Merge(t2 *Taskfile, include *Include) error {
if t1.Env == nil {
t1.Env = &Vars{}
}
t1.Vars.Merge(t2.Vars)
t1.Env.Merge(t2.Env)
t1.Tasks.Merge(t2.Tasks, include)
t1.Vars.Merge(t2.Vars, include)
t1.Env.Merge(t2.Env, include)
t1.Tasks.Merge(t2.Tasks, include, t1.Vars)
return nil
}

View File

@@ -6,6 +6,7 @@ import (
"gopkg.in/yaml.v3"
"github.com/go-task/task/v3/internal/filepathext"
"github.com/go-task/task/v3/internal/omap"
)
@@ -44,7 +45,7 @@ func (t *Tasks) FindMatchingTasks(call *Call) []*MatchingTask {
return matchingTasks
}
func (t1 *Tasks) Merge(t2 Tasks, include *Include) {
func (t1 *Tasks) Merge(t2 Tasks, include *Include, includedTaskfileVars *Vars) {
_ = t2.Range(func(k string, v *Task) error {
// We do a deep copy of the task struct here to ensure that no data can
// be changed elsewhere once the taskfile is merged.
@@ -54,20 +55,25 @@ func (t1 *Tasks) Merge(t2 Tasks, include *Include) {
// taskfile are marked as internal
task.Internal = task.Internal || (include != nil && include.Internal)
// Add namespaces to dependencies, commands and aliases
// Add namespaces to task dependencies
for _, dep := range task.Deps {
if dep != nil && dep.Task != "" {
dep.Task = taskNameWithNamespace(dep.Task, include.Namespace)
}
}
// Add namespaces to task commands
for _, cmd := range task.Cmds {
if cmd != nil && cmd.Task != "" {
cmd.Task = taskNameWithNamespace(cmd.Task, include.Namespace)
}
}
// Add namespaces to task aliases
for i, alias := range task.Aliases {
task.Aliases[i] = taskNameWithNamespace(alias, include.Namespace)
}
// Add namespace aliases
if include != nil {
for _, namespaceAlias := range include.Aliases {
@@ -78,6 +84,15 @@ func (t1 *Tasks) Merge(t2 Tasks, include *Include) {
}
}
if include.AdvancedImport {
task.Dir = filepathext.SmartJoin(include.Dir, task.Dir)
if task.IncludeVars == nil {
task.IncludeVars = &Vars{}
}
task.IncludeVars.Merge(include.Vars, nil)
task.IncludedTaskfileVars = includedTaskfileVars
}
// Add the task to the merged taskfile
taskNameWithNamespace := taskNameWithNamespace(k, include.Namespace)
task.Task = taskNameWithNamespace

View File

@@ -45,11 +45,17 @@ func (vs *Vars) Range(f func(k string, v Var) error) error {
}
// Wrapper around OrderedMap.Merge to ensure we don't get nil pointer errors
func (vs *Vars) Merge(other *Vars) {
func (vs *Vars) Merge(other *Vars, include *Include) {
if vs == nil || other == nil {
return
}
vs.OrderedMap.Merge(other.OrderedMap)
_ = other.Range(func(key string, value Var) error {
if include != nil && include.AdvancedImport {
value.Dir = include.Dir
}
vs.Set(key, value)
return nil
})
}
// Wrapper around OrderedMap.Len to ensure we don't get nil pointer errors
@@ -83,10 +89,10 @@ type Var struct {
}
func (v *Var) UnmarshalYAML(node *yaml.Node) error {
if experiments.AnyVariables.Enabled {
if experiments.MapVariables.Enabled {
// This implementation is not backwards-compatible and replaces the 'sh' key with map variables
if experiments.AnyVariables.Value == "1" {
if experiments.MapVariables.Value == "1" {
var value any
if err := node.Decode(&value); err != nil {
return err
@@ -103,7 +109,7 @@ func (v *Var) UnmarshalYAML(node *yaml.Node) error {
}
// This implementation IS backwards-compatible and keeps the 'sh' key and allows map variables to be added under the `map` key
if experiments.AnyVariables.Value == "2" {
if experiments.MapVariables.Value == "2" {
switch node.Kind {
case yaml.MappingNode:
key := node.Content[0].Value
@@ -141,15 +147,10 @@ func (v *Var) UnmarshalYAML(node *yaml.Node) error {
switch node.Kind {
case yaml.ScalarNode:
var str string
if err := node.Decode(&str); err != nil {
return err
}
v.Value = str
return nil
case yaml.MappingNode:
if len(node.Content) > 2 || node.Content[0].Value != "sh" {
return fmt.Errorf(`task: line %d: maps cannot be assigned to variables`, node.Line)
}
var sh struct {
Sh string
}
@@ -158,7 +159,13 @@ func (v *Var) UnmarshalYAML(node *yaml.Node) error {
}
v.Sh = sh.Sh
return nil
}
return fmt.Errorf("yaml: line %d: cannot unmarshal %s into variable", node.Line, node.ShortTag())
default:
var value any
if err := node.Decode(&value); err != nil {
return err
}
v.Value = value
return nil
}
}

View File

@@ -17,7 +17,6 @@ type Node interface {
Parent() Node
Location() string
Dir() string
Optional() bool
Remote() bool
ResolveEntrypoint(entrypoint string) (string, error)
ResolveDir(dir string) (string, error)
@@ -31,9 +30,8 @@ func NewRootNode(
timeout time.Duration,
) (Node, error) {
dir = getDefaultDir(entrypoint, dir)
// Check if there is something to read on STDIN
stat, _ := os.Stdin.Stat()
if (stat.Mode()&os.ModeCharDevice) == 0 && stat.Size() > 0 {
// If the entrypoint is "-", we read from stdin
if entrypoint == "-" {
return NewStdinNode(dir)
}
return NewNode(l, entrypoint, dir, insecure, timeout)

View File

@@ -2,22 +2,20 @@ package taskfile
type (
NodeOption func(*BaseNode)
// BaseNode is a generic node that implements the Parent() and Optional()
// methods of the NodeReader interface. It does not implement the Read() method
// and it designed to be embedded in other node types so that this boilerplate
// code does not need to be repeated.
// BaseNode is a generic node that implements the Parent() methods of the
// NodeReader interface. It does not implement the Read() method and it
// designed to be embedded in other node types so that this boilerplate code
// does not need to be repeated.
BaseNode struct {
parent Node
optional bool
dir string
parent Node
dir string
}
)
func NewBaseNode(dir string, opts ...NodeOption) *BaseNode {
node := &BaseNode{
parent: nil,
optional: false,
dir: dir,
parent: nil,
dir: dir,
}
// Apply options
@@ -38,16 +36,6 @@ func (node *BaseNode) Parent() Node {
return node.parent
}
func WithOptional(optional bool) NodeOption {
return func(node *BaseNode) {
node.optional = optional
}
}
func (node *BaseNode) Optional() bool {
return node.optional
}
func (node *BaseNode) Dir() string {
return node.dir
}

View File

@@ -6,9 +6,12 @@ import (
"os"
"time"
"github.com/dominikbraun/graph"
"golang.org/x/sync/errgroup"
"gopkg.in/yaml.v3"
"github.com/go-task/task/v3/errors"
"github.com/go-task/task/v3/internal/compiler"
"github.com/go-task/task/v3/internal/filepathext"
"github.com/go-task/task/v3/internal/logger"
"github.com/go-task/task/v3/internal/templater"
@@ -24,33 +27,83 @@ Continue?`
Continue?`
)
// Read reads a Read for a given directory
// Uses current dir when dir is left empty. Uses Read.yml
// or Read.yaml when entrypoint is left empty
func Read(
// A Reader will recursively read Taskfiles from a given source using a directed
// acyclic graph (DAG).
type Reader struct {
graph *ast.TaskfileGraph
node Node
insecure bool
download bool
offline bool
timeout time.Duration
tempDir string
logger *logger.Logger
}
func NewReader(
node Node,
insecure bool,
download bool,
offline bool,
timeout time.Duration,
tempDir string,
l *logger.Logger,
) (*ast.Taskfile, error) {
var _taskfile func(Node) (*ast.Taskfile, error)
_taskfile = func(node Node) (*ast.Taskfile, error) {
tf, err := readTaskfile(node, download, offline, timeout, tempDir, l)
if err != nil {
return nil, err
}
logger *logger.Logger,
) *Reader {
return &Reader{
graph: ast.NewTaskfileGraph(),
node: node,
insecure: insecure,
download: download,
offline: offline,
timeout: timeout,
tempDir: tempDir,
logger: logger,
}
}
// Check that the Taskfile is set and has a schema version
if tf == nil || tf.Version == nil {
return nil, &errors.TaskfileVersionCheckError{URI: node.Location()}
}
func (r *Reader) Read() (*ast.TaskfileGraph, error) {
// Recursively loop through each Taskfile, adding vertices/edges to the graph
if err := r.include(r.node); err != nil {
return nil, err
}
err = tf.Includes.Range(func(namespace string, include ast.Include) error {
cache := &templater.Cache{Vars: tf.Vars}
include = ast.Include{
return r.graph, nil
}
func (r *Reader) include(node Node) error {
// Create a new vertex for the Taskfile
vertex := &ast.TaskfileVertex{
URI: node.Location(),
Taskfile: nil,
}
// Add the included Taskfile to the DAG
// If the vertex already exists, we return early since its Taskfile has
// already been read and its children explored
if err := r.graph.AddVertex(vertex); err == graph.ErrVertexAlreadyExists {
return nil
} else if err != nil {
return err
}
// Read and parse the Taskfile from the file and add it to the vertex
var err error
vertex.Taskfile, err = r.readNode(node)
if err != nil {
return err
}
// Create an error group to wait for all included Taskfiles to be read
var g errgroup.Group
// Loop over each included taskfile
_ = vertex.Taskfile.Includes.Range(func(namespace string, include *ast.Include) error {
vars := compiler.GetEnviron()
vars.Merge(vertex.Taskfile.Vars, nil)
// Start a goroutine to process each included Taskfile
g.Go(func() error {
cache := &templater.Cache{Vars: vars}
include = &ast.Include{
Namespace: include.Namespace,
Taskfile: templater.Replace(include.Taskfile, cache),
Dir: templater.Replace(include.Dir, cache),
@@ -69,14 +122,13 @@ func Read(
return err
}
dir, err := node.ResolveDir(include.Dir)
include.Dir, err = node.ResolveDir(include.Dir)
if err != nil {
return err
}
includeReaderNode, err := NewNode(l, entrypoint, dir, insecure, timeout,
includeNode, err := NewNode(r.logger, entrypoint, include.Dir, r.insecure, r.timeout,
WithParent(node),
WithOptional(include.Optional),
)
if err != nil {
if include.Optional {
@@ -85,106 +137,72 @@ func Read(
return err
}
if err := checkCircularIncludes(includeReaderNode); err != nil {
// Recurse into the included Taskfile
if err := r.include(includeNode); err != nil {
return err
}
includedTaskfile, err := _taskfile(includeReaderNode)
if err != nil {
if include.Optional {
return nil
}
return err
// Create an edge between the Taskfiles
r.graph.Lock()
defer r.graph.Unlock()
edge, err := r.graph.Edge(node.Location(), includeNode.Location())
if err == graph.ErrEdgeNotFound {
// If the edge doesn't exist, create it
err = r.graph.AddEdge(
node.Location(),
includeNode.Location(),
graph.EdgeData([]*ast.Include{include}),
graph.EdgeWeight(1),
)
} else {
// If the edge already exists
edgeData := append(edge.Properties.Data.([]*ast.Include), include)
err = r.graph.UpdateEdge(
node.Location(),
includeNode.Location(),
graph.EdgeData(edgeData),
graph.EdgeWeight(len(edgeData)),
)
}
if len(includedTaskfile.Dotenv) > 0 {
return ErrIncludedTaskfilesCantHaveDotenvs
}
if include.AdvancedImport {
// nolint: errcheck
includedTaskfile.Vars.Range(func(k string, v ast.Var) error {
o := v
o.Dir = dir
includedTaskfile.Vars.Set(k, o)
return nil
})
// nolint: errcheck
includedTaskfile.Env.Range(func(k string, v ast.Var) error {
o := v
o.Dir = dir
includedTaskfile.Env.Set(k, o)
return nil
})
for _, task := range includedTaskfile.Tasks.Values() {
task.Dir = filepathext.SmartJoin(dir, task.Dir)
if task.IncludeVars == nil {
task.IncludeVars = &ast.Vars{}
}
task.IncludeVars.Merge(include.Vars)
task.IncludedTaskfileVars = includedTaskfile.Vars
if errors.Is(err, graph.ErrEdgeCreatesCycle) {
return errors.TaskfileCycleError{
Source: node.Location(),
Destination: includeNode.Location(),
}
}
if err = tf.Merge(includedTaskfile, &include); err != nil {
return err
}
return nil
return err
})
if err != nil {
return nil, err
}
return nil
})
for _, task := range tf.Tasks.Values() {
// If the task is not defined, create a new one
if task == nil {
task = &ast.Task{}
}
// Set the location of the taskfile for each task
if task.Location.Taskfile == "" {
task.Location.Taskfile = tf.Location
}
}
return tf, nil
}
return _taskfile(node)
// Wait for all the go routines to finish
return g.Wait()
}
func readTaskfile(
node Node,
download,
offline bool,
timeout time.Duration,
tempDir string,
l *logger.Logger,
) (*ast.Taskfile, error) {
func (r *Reader) readNode(node Node) (*ast.Taskfile, error) {
var b []byte
var err error
var cache *Cache
if node.Remote() {
cache, err = NewCache(tempDir)
cache, err = NewCache(r.tempDir)
if err != nil {
return nil, err
}
}
// If the file is remote and we're in offline mode, check if we have a cached copy
if node.Remote() && offline {
if node.Remote() && r.offline {
if b, err = cache.read(node); errors.Is(err, os.ErrNotExist) {
return nil, &errors.TaskfileCacheNotFoundError{URI: node.Location()}
} else if err != nil {
return nil, err
}
l.VerboseOutf(logger.Magenta, "task: [%s] Fetched cached copy\n", node.Location())
r.logger.VerboseOutf(logger.Magenta, "task: [%s] Fetched cached copy\n", node.Location())
} else {
downloaded := false
ctx, cf := context.WithTimeout(context.Background(), timeout)
ctx, cf := context.WithTimeout(context.Background(), r.timeout)
defer cf()
// Read the file
@@ -192,16 +210,16 @@ func readTaskfile(
// If we timed out then we likely have a network issue
if node.Remote() && errors.Is(ctx.Err(), context.DeadlineExceeded) {
// If a download was requested, then we can't use a cached copy
if download {
return nil, &errors.TaskfileNetworkTimeoutError{URI: node.Location(), Timeout: timeout}
if r.download {
return nil, &errors.TaskfileNetworkTimeoutError{URI: node.Location(), Timeout: r.timeout}
}
// Search for any cached copies
if b, err = cache.read(node); errors.Is(err, os.ErrNotExist) {
return nil, &errors.TaskfileNetworkTimeoutError{URI: node.Location(), Timeout: timeout, CheckedCache: true}
return nil, &errors.TaskfileNetworkTimeoutError{URI: node.Location(), Timeout: r.timeout, CheckedCache: true}
} else if err != nil {
return nil, err
}
l.VerboseOutf(logger.Magenta, "task: [%s] Network timeout. Fetched cached copy\n", node.Location())
r.logger.VerboseOutf(logger.Magenta, "task: [%s] Network timeout. Fetched cached copy\n", node.Location())
} else if err != nil {
return nil, err
} else {
@@ -210,7 +228,7 @@ func readTaskfile(
// If the node was remote, we need to check the checksum
if node.Remote() && downloaded {
l.VerboseOutf(logger.Magenta, "task: [%s] Fetched remote copy\n", node.Location())
r.logger.VerboseOutf(logger.Magenta, "task: [%s] Fetched remote copy\n", node.Location())
// Get the checksums
checksum := checksum(b)
@@ -225,7 +243,7 @@ func readTaskfile(
prompt = fmt.Sprintf(taskfileChangedPrompt, node.Location())
}
if prompt != "" {
if err := l.Prompt(logger.Yellow, prompt, "n", "y", "yes"); err != nil {
if err := r.logger.Prompt(logger.Yellow, prompt, "n", "y", "yes"); err != nil {
return nil, &errors.TaskfileNotTrustedError{URI: node.Location()}
}
}
@@ -237,7 +255,7 @@ func readTaskfile(
return nil, err
}
// Cache the file
l.VerboseOutf(logger.Magenta, "task: [%s] Caching downloaded file\n", node.Location())
r.logger.VerboseOutf(logger.Magenta, "task: [%s] Caching downloaded file\n", node.Location())
if err = cache.write(node, b); err != nil {
return nil, err
}
@@ -249,29 +267,19 @@ func readTaskfile(
if err := yaml.Unmarshal(b, &t); err != nil {
return nil, &errors.TaskfileInvalidError{URI: filepathext.TryAbsToRel(node.Location()), Err: err}
}
// Set the taskfile/task's locations
t.Location = node.Location()
for _, task := range t.Tasks.Values() {
// If the task is not defined, create a new one
if task == nil {
task = &ast.Task{}
}
// Set the location of the taskfile for each task
if task.Location.Taskfile == "" {
task.Location.Taskfile = t.Location
}
}
return &t, nil
}
func checkCircularIncludes(node Node) error {
if node == nil {
return errors.New("task: failed to check for include cycle: node was nil")
}
if node.Parent() == nil {
return errors.New("task: failed to check for include cycle: node.Parent was nil")
}
curNode := node
location := node.Location()
for curNode.Parent() != nil {
curNode = curNode.Parent()
curLocation := curNode.Location()
if curLocation == location {
return fmt.Errorf("task: include cycle detected between %s <--> %s",
curLocation,
node.Parent().Location(),
)
}
}
return nil
}

View File

@@ -16,9 +16,6 @@ import (
)
var (
// ErrIncludedTaskfilesCantHaveDotenvs is returned when a included Taskfile contains dotenvs
ErrIncludedTaskfilesCantHaveDotenvs = errors.New("task: Included Taskfiles can't have dotenv declarations. Please, move the dotenv declaration to the main Taskfile")
defaultTaskfiles = []string{
"Taskfile.yml",
"taskfile.yml",
@@ -29,7 +26,6 @@ var (
"Taskfile.dist.yaml",
"taskfile.dist.yaml",
}
allowedContentTypes = []string{
"text/plain",
"text/yaml",

View File

@@ -14,6 +14,7 @@ tasks:
cmds:
- task: local
- task: global
- task: multiple_type
local:
vars:
@@ -31,3 +32,11 @@ tasks:
BAR: overriden
cmds:
- echo "FOO='$FOO' BAR='$BAR' BAZ='$BAZ'" > global.txt
multiple_type:
env:
FOO: 1
BAR: true
BAZ: 1.1
cmds:
- echo "FOO='$FOO' BAR='$BAR' BAZ='$BAZ'" > multiple_type.txt

View File

@@ -2,15 +2,15 @@ version: "3"
includes:
included1:
taskfile: include/Taskfile.include.yml
taskfile: include/Taskfile.include1.yml
vars:
VAR_1: included1-var1
included2:
taskfile: include/Taskfile.include.yml
taskfile: include/Taskfile.include2.yml
vars:
VAR_1: included2-var1
included3:
taskfile: include/Taskfile.include.yml
taskfile: include/Taskfile.include3.yml
tasks:
task1:

View File

@@ -0,0 +1,11 @@
version: "3"
vars:
VAR_1: '{{.VAR_1 | default "included-default-var1"}}'
VAR_2: '{{.VAR_2 | default "included-default-var2"}}'
tasks:
task1:
cmds:
- echo "VAR_1 is {{.VAR_1}}"
- echo "VAR_2 is {{.VAR_2}}"

View File

@@ -0,0 +1,11 @@
version: "3"
vars:
VAR_1: '{{.VAR_1 | default "included-default-var1"}}'
VAR_2: '{{.VAR_2 | default "included-default-var2"}}'
tasks:
task1:
cmds:
- echo "VAR_1 is {{.VAR_1}}"
- echo "VAR_2 is {{.VAR_2}}"

View File

@@ -1,10 +0,0 @@
version: "3"
vars:
MODULE_NAME: included
includes:
include: './{{.MODULE_NAME}}/Taskfile.yml'
include-with-dir:
taskfile: './{{.MODULE_NAME}}/Taskfile.yml'
dir: '{{.MODULE_NAME}}'

View File

@@ -0,0 +1,7 @@
version: "3"
vars:
MODULE_NAME: included
includes:
include: '../{{.MODULE_NAME}}/Taskfile.yml'

View File

@@ -0,0 +1,9 @@
version: "3"
vars:
MODULE_NAME: included
includes:
include-with-dir:
taskfile: '../{{.MODULE_NAME}}/Taskfile.yml'
dir: '../{{.MODULE_NAME}}'

View File

@@ -0,0 +1,4 @@
version: "3"
includes:
include-with-env-variable: '../{{.MODULE}}/Taskfile.yml'

View File

@@ -8,6 +8,7 @@ tasks:
- task: ref
- task: ref-sh
- task: ref-dep
- task: ref-resolver
- task: json
- task: yaml
@@ -16,10 +17,10 @@ tasks:
MAP:
map: {"name":"Alice","age":30,"children":[{"name":"Bob","age":5},{"name":"Charlie","age":3},{"name":"Diane","age":1}]}
cmds:
- task: print-var
- task: print-story
vars:
VAR:
ref: MAP
ref: .MAP
nested-map:
vars:
@@ -44,12 +45,12 @@ tasks:
MAP:
map: {"name":"Alice","age":30,"children":[{"name":"Bob","age":5},{"name":"Charlie","age":3},{"name":"Diane","age":1}]}
MAP_REF:
ref: MAP
ref: .MAP
cmds:
- task: print-var
- task: print-story
vars:
VAR:
ref: MAP_REF
ref: .MAP_REF
ref-sh:
vars:
@@ -58,22 +59,34 @@ tasks:
JSON:
json: "{{.JSON_STRING}}"
MAP_REF:
ref: JSON
ref: .JSON
cmds:
- task: print-var
- task: print-story
vars:
VAR:
ref: MAP_REF
ref: .MAP_REF
ref-dep:
vars:
MAP:
map: {"name":"Alice","age":30,"children":[{"name":"Bob","age":5},{"name":"Charlie","age":3},{"name":"Diane","age":1}]}
deps:
- task: print-story
vars:
VAR:
ref: .MAP
ref-resolver:
vars:
MAP:
map: {"name":"Alice","age":30,"children":[{"name":"Bob","age":5},{"name":"Charlie","age":3},{"name":"Diane","age":1}]}
MAP_REF:
ref: .MAP
cmds:
- task: print-var
vars:
VAR:
ref: MAP
ref: (index .MAP_REF.children 0).name
json:
vars:
@@ -82,10 +95,10 @@ tasks:
JSON:
json: "{{.JSON_STRING}}"
cmds:
- task: print-var
- task: print-story
vars:
VAR:
ref: JSON
ref: .JSON
yaml:
vars:
@@ -94,12 +107,16 @@ tasks:
YAML:
yaml: "{{.YAML_STRING}}"
cmds:
- task: print-var
- task: print-story
vars:
VAR:
ref: YAML
ref: .YAML
print-var:
cmds:
- echo "{{.VAR}}"
print-story:
cmds:
- >-
echo "{{.VAR.name}} has {{len .VAR.children}} children called

View File

@@ -104,9 +104,9 @@ func (e *Executor) compiledTask(call *ast.Call, evaluateShVars bool) (*ast.Task,
}
new.Env = &ast.Vars{}
new.Env.Merge(templater.ReplaceVars(e.Taskfile.Env, cache))
new.Env.Merge(templater.ReplaceVars(dotenvEnvs, cache))
new.Env.Merge(templater.ReplaceVars(origTask.Env, cache))
new.Env.Merge(templater.ReplaceVars(e.Taskfile.Env, cache), nil)
new.Env.Merge(templater.ReplaceVars(dotenvEnvs, cache), nil)
new.Env.Merge(templater.ReplaceVars(origTask.Env, cache), nil)
if evaluateShVars {
err = new.Env.Range(func(k string, v ast.Var) error {
// If the variable is not dynamic, we can set it and return
@@ -164,17 +164,6 @@ func (e *Executor) compiledTask(call *ast.Call, evaluateShVars bool) (*ast.Task,
newCmd.Cmd = templater.Replace(cmd.Cmd, cache)
newCmd.Task = templater.Replace(cmd.Task, cache)
newCmd.Vars = templater.ReplaceVars(cmd.Vars, cache)
// Loop over the command's variables and resolve any references to other variables
err := cmd.Vars.Range(func(k string, v ast.Var) error {
if v.Ref != "" {
refVal := vars.Get(v.Ref)
newCmd.Vars.Set(k, refVal)
}
return nil
})
if err != nil {
return nil, err
}
new.Cmds = append(new.Cmds, newCmd)
}
}
@@ -214,17 +203,6 @@ func (e *Executor) compiledTask(call *ast.Call, evaluateShVars bool) (*ast.Task,
newDep := dep.DeepCopy()
newDep.Task = templater.Replace(dep.Task, cache)
newDep.Vars = templater.ReplaceVars(dep.Vars, cache)
// Loop over the dep's variables and resolve any references to other variables
err := dep.Vars.Range(func(k string, v ast.Var) error {
if v.Ref != "" {
refVal := vars.Get(v.Ref)
newDep.Vars.Set(k, refVal)
}
return nil
})
if err != nil {
return nil, err
}
new.Deps = append(new.Deps, newDep)
}
}

View File

@@ -16,7 +16,7 @@ communicate these kinds of thoughts to the community. So, with that in mind,
this is the first (hopefully of many) blog posts talking about Task and what
we're up to.
<!--truncate-->
{/* truncate */}
## :calendar: So, what have we been up to?
@@ -122,7 +122,7 @@ I plan to write more of these blog posts in the future on a variety of
Task-related topics, so make sure to check in occasionally and see what we're up
to!
<!-- prettier-ignore-start -->
{/* prettier-ignore-start */}
[vscode-task]: https://github.com/go-task/vscode-task
[crowdin]: https://crowdin.com
[contributors]: https://github.com/go-task/task/graphs/contributors
@@ -139,4 +139,4 @@ to!
[experiments-project]: https://github.com/orgs/go-task/projects/1
[gentle-force-experiment]: https://github.com/go-task/task/issues/1200
[remote-taskfiles-experiment]: https://github.com/go-task/task/issues/1317
<!-- prettier-ignore-end -->
{/* prettier-ignore-end */}

View File

@@ -0,0 +1,181 @@
---
title: Any Variables
description: Task variables are no longer limited to strings!
slug: any-variables
authors: [pd93]
tags: [experiments, variables]
image: https://i.imgur.com/mErPwqL.png
hide_table_of_contents: false
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
Task has always had variables, but even though you were able to define them
using different YAML types, they would always be converted to strings by Task.
This limited users to string manipulation and encouraged messy workarounds for
simple problems. Starting from [v3.37.0][v3.37.0], this is no longer the case!
Task now supports most variable types, including **booleans**, **integers**,
**floats** and **arrays**!
{/* truncate */}
## What's the big deal?
These changes allow you to use variables in a much more natural way and opens up
a wide variety of sprig functions that were previously useless. Take a look at
some of the examples below for some inspiration.
### Evaluating booleans
No more comparing strings to "true" or "false". Now you can use actual boolean
values in your templates:
<Tabs defaultValue="2"
values={[
{label: 'Before', value: '1'},
{label: 'After', value: '2'}
]}>
<TabItem value="1">
```yaml
version: 3
tasks:
foo:
vars:
BOOL: true # <-- Parsed as a string even though its a YAML boolean
cmds:
- '{{if eq .BOOL "true"}}echo foo{{end}}'
```
</TabItem>
<TabItem value="2">
```yaml
version: 3
tasks:
foo:
vars:
BOOL: true # <-- Parsed as a boolean
cmds:
- '{{if .BOOL}}echo foo{{end}}' # <-- No need to compare to "true"
```
</TabItem></Tabs>
### Arithmetic
You can now perform basic arithmetic operations on integer and float variables:
```yaml
version: 3
tasks:
foo:
vars:
INT: 10
FLOAT: 3.14159
cmds:
- 'echo {{add .INT .FLOAT}}'
```
You can use any of the following arithmetic functions: `add`, `sub`, `mul`,
`div`, `mod`, `max`, `min`, `floor`, `ceil`, `round` and `randInt`. Check out
the [slim-sprig math documentation][slim-sprig-math] for more information.
### Arrays
You can now range over arrays inside templates and use list-based functions:
```yaml
version: 3
tasks:
foo:
vars:
ARRAY: [1, 2, 3]
cmds:
- 'echo {{range .ARRAY}}{{.}}{{end}}'
```
You can use any of the following list-based functions: `first`, `rest`, `last`,
`initial`, `append`, `prepend`, `concat`, `reverse`, `uniq`, `without`, `has`,
`compact`, `slice` and `chunk`. Check out the [slim-sprg lists
documentation][slim-sprig-list] for more information.
### Looping over variables using `for`
Previously, you would have to use a delimiter separated string to loop over an
arbitrary list of items in a variable and split them by using the `split` subkey
to specify the delimiter. However, we have now added support for looping over
"collection-type" variables using the `for` keyword, so now you are able to loop
over list variables directly:
<Tabs defaultValue="2"
values={[
{label: 'Before', value: '1'},
{label: 'After', value: '2'}
]}>
<TabItem value="1">
```yaml
version: 3
tasks:
foo:
vars:
LIST: 'foo,bar,baz'
cmds:
- for:
var: LIST
split: ','
cmd: echo {{.ITEM}}
```
</TabItem>
<TabItem value="2">
```yaml
version: 3
tasks:
foo:
vars:
LIST: ['foo', 'bar', 'baz']
cmds:
- for:
var: LIST
cmd: echo {{.ITEM}}
```
</TabItem></Tabs>
## What about maps?
Maps were originally included in the Any Variables experiment. However, they
weren't quite ready yet. Instead of making you wait for everything to be ready
at once, we have released support for all other variable types and we will
continue working on map support in the new "[Map Variables][map-variables]"
experiment.
:::note
If you were previously using maps with the Any Variables experiment and wish to
continue using them, you will need to enable the new [Map Variables
experiment][map-variables] instead.
:::
We're looking for feedback on a couple of different proposals, so please give
them a go and let us know what you think. :pray:
{/* prettier-ignore-start */}
[v3.37.0]: https://github.com/go-task/task/releases/tag/v3.37.0
[slim-sprig-math]: https://go-task.github.io/slim-sprig/math.html
[slim-sprig-list]: https://go-task.github.io/slim-sprig/lists.html
[map-variables]: /experiments/map-variables
{/* prettier-ignore-end */}

View File

@@ -266,7 +266,7 @@ vars:
| `prefix` | `string` | | Defines a string to prefix the output of tasks running in parallel. Only used when the output mode is `prefixed`. |
| `ignore_error` | `bool` | `false` | Continue execution if errors happen while executing commands. |
| `run` | `string` | The one declared globally in the Taskfile or `always` | Specifies whether the task should run again or not if called more than once. Available options: `always`, `once` and `when_changed`. |
| `platforms` | `[]string` | All platforms | Specifies which platforms the task should be run on. [Valid GOOS and GOARCH values allowed](https://github.com/golang/go/blob/main/src/go/build/syslist.go). Task will be skipped otherwise. |
| `platforms` | `[]string` | All platforms | Specifies which platforms the task should be run on. [Valid GOOS and GOARCH values allowed](https://github.com/golang/go/blob/master/src/go/build/syslist.go). Task will be skipped otherwise. |
| `set` | `[]string` | | Specify options for the [`set` builtin](https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html). |
| `shopt` | `[]string` | | Specify option for the [`shopt` builtin](https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html). |
@@ -300,7 +300,7 @@ tasks:
| `vars` | [`map[string]Variable`](#variable) | | Optional additional variables to be passed to the referenced task. Only relevant when setting `task` instead of `cmd`. |
| `ignore_error` | `bool` | `false` | Continue execution if errors happen while executing the command. |
| `defer` | `string` | | Alternative to `cmd`, but schedules the command to be executed at the end of this task instead of immediately. This cannot be used together with `cmd`. |
| `platforms` | `[]string` | All platforms | Specifies which platforms the command should be run on. [Valid GOOS and GOARCH values allowed](https://github.com/golang/go/blob/main/src/go/build/syslist.go). Command will be skipped otherwise. |
| `platforms` | `[]string` | All platforms | Specifies which platforms the command should be run on. [Valid GOOS and GOARCH values allowed](https://github.com/golang/go/blob/master/src/go/build/syslist.go). Command will be skipped otherwise. |
| `set` | `[]string` | | Specify options for the [`set` builtin](https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html). |
| `shopt` | `[]string` | | Specify option for the [`shopt` builtin](https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html). |

View File

@@ -5,6 +5,32 @@ sidebar_position: 14
# Changelog
## v3.37.1 - 2024-05-09
- Fix bug where non-string values (numbers, bools) added to `env:` weren't been
correctly exported (#1640, #1641 by @vmaerten and @andreynering).
## v3.37.0 - 2024-05-08
- Released the
[Any Variables experiment](https://taskfile.dev/blog/any-variables), but
[_without support for maps_](https://github.com/go-task/task/issues/1415#issuecomment-2044756925)
(#1415, #1547 by @pd93).
- Refactored how Task reads, parses and merges Taskfiles using a DAG (#1563,
#1607 by @pd93).
- Fix a bug which stopped tasks from using `stdin` as input (#1593, #1623 by
@pd03).
- Fix error when a file or directory in the project contained a special char
like `&`, `(` or `)` (#1551, #1584 by @andreynering).
- Added alias `q` for template function `shellQuote` (#1601, #1603 by @vergenzt)
- Added support for `~` on ZSH completions (#1613 by @jwater7).
- Added the ability to pass variables by reference using Go template syntax when
the
[Map Variables experiment](https://taskfile.dev/experiments/map-variables/) is
enabled (#1612 by @pd93).
- Added support for environment variables in the templating engine in `includes`
(#1610 by @vmaerten).
## v3.36.0 - 2024-04-08
- Added support for

View File

@@ -45,5 +45,5 @@ if you want to adopt the new behavior, you can continue to use the `--force`
flag as you do now!
{/* prettier-ignore-start */}
[enabling-experiments]: /experiments/#enabling-experiments
[enabling-experiments]: ./experiments.mdx#enabling-experiments
{/* prettier-ignore-end */}

View File

@@ -1,11 +1,11 @@
---
slug: /experiments/any-variables/
slug: /experiments/map-variables/
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Any Variables (#1415)
# Map Variables (#1585)
:::caution
@@ -15,19 +15,9 @@ environment. They are intended for testing and feedback only.
:::
Currently, Task only supports string variables. This experiment allows you to
specify and use the following variable types:
- `string`
- `bool`
- `int`
- `float`
- `array`
- `map`
This allows you to have a lot more flexibility in how you use variables in
Task's templating engine. There are two active proposals for this experiment.
Click on the tabs below to switch between them.
Currently, Task supports all variable types except for maps. This experiment
adds two different proposals for map variables. Click on the tabs below to
switch between them.
<Tabs defaultValue="1" queryString="proposal"
values={[
@@ -48,13 +38,11 @@ This experiment proposal breaks the following functionality:
:::info
To enable this experiment, set the environment variable:
`TASK_X_ANY_VARIABLES=1`. Check out [our guide to enabling experiments
`TASK_X_MAP_VARIABLES=1`. Check out [our guide to enabling experiments
][enabling-experiments] for more information.
:::
## Maps
This proposal removes support for the `sh` keyword in favour of a new syntax for
dynamically defined variables, This allows you to define a map directly as you
would for any other type:
@@ -111,19 +99,16 @@ will now need to escape the `$` with a backslash (`\`) to stop Task from
executing it as a command.
</TabItem>
<TabItem value="2">
:::info
To enable this experiment, set the environment variable:
`TASK_X_ANY_VARIABLES=2`. Check out [our guide to enabling experiments
`TASK_X_MAP_VARIABLES=2`. Check out [our guide to enabling experiments
][enabling-experiments] for more information.
:::
## Maps
This proposal maintains backwards-compatibility and the `sh` subkey and adds
another new `map` subkey for defining map variables:
@@ -150,7 +135,13 @@ objects/arrays. This is similar to the `fromJSON` template function, but means
that you only have to parse the JSON/YAML once when you declare the variable,
instead of every time you want to access a value.
Before:
<Tabs defaultValue="2"
values={[
{label: 'Before', value: '1'},
{label: 'After', value: '2'}
]}>
<TabItem value="1">
```yaml
version: 3
@@ -164,7 +155,8 @@ tasks:
- 'echo {{(fromJSON .FOO).b}}'
```
After:
</TabItem>
<TabItem value="2">
```yaml
version: 3
@@ -179,12 +171,26 @@ tasks:
- 'echo {{.FOO.b}}'
```
</TabItem></Tabs>
## Variables by reference
Lastly, this proposal adds support for defining and passing variables by
reference. This is really important now that variables can be types other than a
string. Previously, to send a variable from one task to another, you would have
to use the templating system to pass it:
string.
Previously, to send a variable from one task to another, you would have to use
the templating system. Unfortunately, the templater _always_ outputs a string
and operations on the passed variable may not have behaved as expected. With
this proposal, you can now pass variables by reference using the `ref` subkey:
<Tabs defaultValue="2"
values={[
{label: 'Before', value: '1'},
{label: 'After', value: '2'}
]}>
<TabItem value="1">
```yaml
version: 3
@@ -202,10 +208,8 @@ tasks:
- 'echo {{index .FOO 0}}' # <-- FOO is a string so the task outputs '91' which is the ASCII code for '[' instead of the expected 'A'
```
Unfortunately, this results in the value always being passed as a string as this
is the output type of the templater and operations on the passed variable may
not behave as expected. With this proposal, you can now pass variables by
reference using the `ref` subkey:
</TabItem>
<TabItem value="2">
```yaml
version: 3
@@ -218,12 +222,14 @@ tasks:
- task: bar
vars:
FOO:
ref: FOO # <-- FOO gets passed by reference to bar and maintains its type
ref: .FOO # <-- FOO gets passed by reference to bar and maintains its type
bar:
cmds:
- 'echo {{index .FOO 0}}' # <-- FOO is still a map so the task outputs 'A' as expected
```
</TabItem></Tabs>
This means that the type of the variable is maintained when it is passed to
another Task. This also works the same way when calling `deps` and when defining
a variable and can be used in any combination:
@@ -236,27 +242,54 @@ tasks:
vars:
FOO: [A, B, C] # <-- FOO is defined as an array
BAR:
ref: FOO # <-- BAR is defined as a reference to FOO
ref: .FOO # <-- BAR is defined as a reference to FOO
deps:
- task: bar
vars:
BAR:
ref: BAR # <-- BAR gets passed by reference to bar and maintains its type
ref: .BAR # <-- BAR gets passed by reference to bar and maintains its type
bar:
cmds:
- 'echo {{index .BAR 0}}' # <-- BAR still refers to FOO so the task outputs 'A'
```
All references use the same templating syntax as regular templates, so in
addition to simply calling `.FOO`, you can also pass subkeys (`.FOO.BAR`) or
indexes (`index .FOO 0`) and use functions (`len .FOO`):
```yaml
version: 3
tasks:
foo:
vars:
FOO: [A, B, C] # <-- FOO is defined as an array
cmds:
- task: bar
vars:
FOO:
ref: index .FOO 0 # <-- The element at index 0 is passed by reference to bar
bar:
cmds:
- 'echo {{.MYVAR}}' # <-- FOO is just the letter 'A'
```
</TabItem></Tabs>
---
## Looping over maps
## Common to both proposals
This experiment also adds support for looping over maps using the `for` keyword,
just like arrays. In addition to the `{{.ITEM}}` variable being populated when
looping over a map, we also make an additional `{{.KEY}}` variable available
that holds the string value of the map key.
Both proposals add support for all other variable types by directly defining
them in the Taskfile. For example:
<Tabs defaultValue="1" queryString="proposal"
values={[
{label: 'Proposal 1', value: '1'},
{label: 'Proposal 2', value: '2'}
]}>
### Evaluating booleans
<TabItem value="1">
```yaml
version: 3
@@ -264,64 +297,15 @@ version: 3
tasks:
foo:
vars:
BOOL: false
cmds:
- '{{if .BOOL}}echo foo{{end}}'
```
### Arithmetic
```yaml
version: 3
tasks:
foo:
vars:
INT: 10
FLOAT: 3.14159
cmds:
- 'echo {{add .INT .FLOAT}}'
```
### Ranging
```yaml
version: 3
tasks:
foo:
vars:
ARRAY: [1, 2, 3]
cmds:
- 'echo {{range .ARRAY}}{{.}}{{end}}'
```
There are many more templating functions which can be used with the new types of
variables. For a full list, see the [slim-sprig][slim-sprig] documentation.
## Looping over variables
Previously, you would have to use a delimiter separated string to loop over an
arbitrary list of items in a variable and split them by using the `split` subkey
to specify the delimiter:
```yaml
version: 3
tasks:
foo:
vars:
LIST: 'foo,bar,baz'
MAP: {a: 1, b: 2, c: 3}
cmds:
- for:
var: LIST
split: ','
cmd: echo {{.ITEM}}
var: MAP
cmd: 'echo "{{.KEY}}: {{.ITEM}}"'
```
Both of these proposals add support for looping over "collection-type" variables
using the `for` keyword, so now you are able to loop over a map/array variable
directly:
</TabItem>
<TabItem value="2">
```yaml
version: 3
@@ -329,18 +313,23 @@ version: 3
tasks:
foo:
vars:
LIST: [foo, bar, baz]
map:
MAP: {a: 1, b: 2, c: 3}
cmds:
- for:
var: LIST
cmd: echo {{.ITEM}}
var: MAP
cmd: 'echo "{{.KEY}}: {{.ITEM}}"'
```
When looping over a map we also make an additional `{{.KEY}}` variable availabe
that holds the string value of the map key. Remember that maps are unordered, so
:::note
Remember that maps are unordered, so
the order in which the items are looped over is random.
:::
</TabItem></Tabs>
{/* prettier-ignore-start */}
[enabling-experiments]: /experiments/#enabling-experiments
[slim-sprig]: https://go-task.github.io/slim-sprig/
[enabling-experiments]: ./experiments.mdx#enabling-experiments
{/* prettier-ignore-end */}

View File

@@ -48,6 +48,20 @@ tasks:
and you run `task my-remote-namespace:hello`, it will print the text: "Hello
from the remote Taskfile!" to your console.
The Taskfile location is processed by the templating system, so you can
reference environment variables in your URL if you need to add authentication.
For example:
```yaml
version: '3'
includes:
my-remote-namespace: https://{{.TOKEN}}@raw.githubusercontent.com/my-org/my-repo/main/Taskfile.yml
```
`TOKEN=my-token task my-remote-namespace:hello` will be resolved by Task to
`https://my-token@raw.githubusercontent.com/my-org/my-repo/main/Taskfile.yml`
## Security
Running commands from sources that you do not control is always a potential
@@ -99,6 +113,6 @@ the `--timeout` flag and specifying a duration. For example, `--timeout 5s` will
set the timeout to 5 seconds.
{/* prettier-ignore-start */}
[enabling-experiments]: /experiments/#enabling-experiments
[enabling-experiments]: ./experiments.mdx#enabling-experiments
[man-in-the-middle-attacks]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
{/* prettier-ignore-end */}

View File

@@ -38,5 +38,5 @@ information.
\{Short explanation of how users should migrate to the new behavior\}
{/* prettier-ignore-start */}
[enabling-experiments]: /experiments/#enabling-experiments
[enabling-experiments]: ./experiments.mdx#enabling-experiments
{/* prettier-ignore-end */}

View File

@@ -31,7 +31,7 @@ brew install go-task
### pkgx
If you're on macOS or Linux and have [pkgx](https://pkgx.sh/) installed, getting Task is as
If you're on macOS or Linux and have [pkgx][pkgx] installed, getting Task is as
simple as running:
```shell
@@ -299,5 +299,5 @@ Invoke-Expression -Command path/to/task.ps1
[godownloader]: https://github.com/goreleaser/godownloader
[choco]: https://chocolatey.org/
[scoop]: https://scoop.sh/
[tea]: https://tea.xyz/
[pkgx]: https://pkgx.sh/
{/* prettier-ignore-end */}

View File

@@ -256,8 +256,8 @@ The variable priority order was also different:
4. `Taskvars.yml` variables
{/* prettier-ignore-start */}
[deprecate-version-2-schema]: /deprecations/version-2-schema/
[output]: /usage#output-syntax
[ignore_errors]: /usage#ignore-errors
[includes]: /usage#including-other-taskfiles
[deprecate-version-2-schema]: ./deprecations/version_2_schema.mdx
[output]: ./usage.mdx#output-syntax
[ignore_errors]: ./usage.mdx#ignore-errors
[includes]: ./usage.mdx#including-other-taskfiles
{/* prettier-ignore-end */}

View File

@@ -121,13 +121,14 @@ tasks:
### Reading a Taskfile from stdin
Taskfile also supports reading from stdin. This is useful if you are generating
Taskfiles dynamically and don't want write them to disk. This works just like
any other program that supports stdin. For example:
Taskfiles dynamically and don't want write them to disk. To tell task to read
from stdin, you must specify the `-t/--taskfile` flag with the special `-`
value. You may then pipe into Task as you would any other program:
```shell
task < <(cat ./Taskfile.yml)
task -t - <(cat ./Taskfile.yml)
# OR
cat ./Taskfile.yml | task
cat ./Taskfile.yml | task -t -
```
## Environment variables
@@ -947,8 +948,26 @@ tasks:
## Variables
When doing interpolation of variables, Task will look for the below. They are
listed below in order of importance (i.e. most important first):
Task allows you to set variables using the `vars` keyword. The following
variable types are supported:
- `string`
- `bool`
- `int`
- `float`
- `array`
:::note
Maps are not supported by default, but there is an
[experiment][map-variables] that can be enabled to add support. If
you're interested in this functionality, we would appreciate your feedback.
:pray:
:::
Variables can be set in many places in a Taskfile. When executing templates,
Task will look for variables in the order listed below (most important first):
- Variables declared in the task definition
- Variables given while calling a task from another (See
@@ -1093,8 +1112,8 @@ tasks:
### Looping over variables
To loop over the contents of a variable, you simply need to specify the variable
you want to loop over. By default, variables will be split on any whitespace
characters.
you want to loop over. By default, string variables will be split on any
whitespace characters.
```yaml
version: '3'
@@ -1108,8 +1127,8 @@ tasks:
cmd: cat {{.ITEM}}
```
If you need to split on a different character, you can do this by specifying the
`split` property:
If you need to split a string on a different character, you can do this by
specifying the `split` property:
```yaml
version: '3'
@@ -1123,6 +1142,26 @@ tasks:
cmd: cat {{.ITEM}}
```
You can also loop over arrays directly (and maps if you have the
[maps experiment](/experiments/map-variables) enabled):
```yaml
version: 3
tasks:
foo:
vars:
LIST: [foo, bar, baz]
cmds:
- for:
var: LIST
cmd: echo {{.ITEM}}
```
When looping over a map we also make an additional `{{.KEY}}` variable available
that holds the string value of the map key. Remember that maps are unordered, so
the order in which the items are looped over is random.
All of this also works with dynamic variables!
```yaml
@@ -1377,7 +1416,7 @@ Task also adds the following functions:
converts a string from `/` path format to `\`.
- `exeExt`: Returns the right executable extension for the current OS (`".exe"`
for Windows, `""` for others).
- `shellQuote`: Quotes a string to make it safe for use in shell scripts. Task
- `shellQuote` (aliased to `q`): Quotes a string to make it safe for use in shell scripts. Task
uses [this Go function](https://pkg.go.dev/mvdan.cc/sh/v3@v3.4.0/syntax#Quote)
for this. The Bash dialect is assumed.
- `splitArgs`: Splits a string as if it were a command's arguments. Task uses
@@ -1956,4 +1995,5 @@ if called by another task, either directly or as a dependency.
{/* prettier-ignore-start */}
[gotemplate]: https://golang.org/pkg/text/template/
[map-variables]: ./experiments/map_variables.mdx
{/* prettier-ignore-end */}

View File

@@ -21,8 +21,8 @@ const config: Config = {
tagline: 'A task runner / simpler Make alternative written in Go ',
url: 'https://taskfile.dev',
baseUrl: '/',
onBrokenLinks: 'throw',
onBrokenMarkdownLinks: 'throw',
onBrokenLinks: 'warn',
onBrokenMarkdownLinks: 'warn',
favicon: 'img/favicon.ico',
organizationName: 'go-task',
@@ -31,56 +31,12 @@ const config: Config = {
i18n: {
defaultLocale: 'en',
locales: [
'en',
'es-ES',
'fr-FR',
'ja-JP',
'pt-BR',
'ru-RU',
'tr-TR',
'zh-Hans'
],
locales: ['en'],
localeConfigs: {
en: {
label: 'English',
direction: 'ltr',
htmlLang: 'en-US'
},
'es-ES': {
label: `Español (${translationProgress['es-ES'] || 0}%)`,
direction: 'ltr',
htmlLang: 'es-ES'
},
'fr-FR': {
label: `Français (${translationProgress['fr'] || 0}%)`,
direction: 'ltr',
htmlLang: 'fr-FR'
},
'ja-JP': {
label: `日本語 (${translationProgress['ja'] || 0}%)`,
direction: 'ltr',
htmlLang: 'ja-JP'
},
'pt-BR': {
label: `Português (${translationProgress['pt-BR'] || 0}%)`,
direction: 'ltr',
htmlLang: 'pt-BR'
},
'ru-RU': {
label: `Pусский (${translationProgress['ru'] || 0}%)`,
direction: 'ltr',
htmlLang: 'ru-RU'
},
'tr-TR': {
label: `Türkçe (${translationProgress['tr'] || 0}%)`,
direction: 'ltr',
htmlLang: 'tr-TR'
},
'zh-Hans': {
label: `简体中文 (${translationProgress['zh-CN'] || 0}%)`,
direction: 'ltr',
htmlLang: 'zh-Hans'
}
}
},
@@ -168,16 +124,6 @@ const config: Config = {
position: 'right',
dropdownActiveClassDisabled: true,
},
{
type: 'localeDropdown',
position: 'right',
dropdownItemsAfter: [
{
to: '/translate/',
label: 'Help Us Translate'
}
]
},
{
href: GITHUB_URL,
position: 'right',

View File

@@ -512,7 +512,7 @@
"type": "object",
"properties": {
"exclude": {
"description": "File or glob patter to exclude from the list",
"description": "File or glob pattern to exclude from the list",
"type": "string"
}
}
@@ -648,7 +648,7 @@
"$ref": "#/definitions/tasks"
},
"silent": {
"description": "Default 'silent' options for this Taskfile. If `false`, can be overidden with `true` in a task by task basis.",
"description": "Default 'silent' options for this Taskfile. If `false`, can be overridden with `true` in a task by task basis.",
"type": "boolean"
},
"set": {

View File

@@ -266,7 +266,7 @@ vars:
| `prefix` | `string` | | Defines a string to prefix the output of tasks running in parallel. Only used when the output mode is `prefixed`. |
| `ignore_error` | `bool` | `false` | Continue execution if errors happen while executing commands. |
| `run` | `string` | The one declared globally in the Taskfile or `always` | Specifies whether the task should run again or not if called more than once. Available options: `always`, `once` and `when_changed`. |
| `platforms` | `[]string` | All platforms | Specifies which platforms the task should be run on. [Valid GOOS and GOARCH values allowed](https://github.com/golang/go/blob/main/src/go/build/syslist.go). Task will be skipped otherwise. |
| `platforms` | `[]string` | All platforms | Specifies which platforms the task should be run on. [Valid GOOS and GOARCH values allowed](https://github.com/golang/go/blob/master/src/go/build/syslist.go). Task will be skipped otherwise. |
| `set` | `[]string` | | Specify options for the [`set` builtin](https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html). |
| `shopt` | `[]string` | | Specify option for the [`shopt` builtin](https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html). |
@@ -300,7 +300,7 @@ tasks:
| `vars` | [`map[string]Variable`](#variable) | | Optional additional variables to be passed to the referenced task. Only relevant when setting `task` instead of `cmd`. |
| `ignore_error` | `bool` | `false` | Continue execution if errors happen while executing the command. |
| `defer` | `string` | | Alternative to `cmd`, but schedules the command to be executed at the end of this task instead of immediately. This cannot be used together with `cmd`. |
| `platforms` | `[]string` | All platforms | Specifies which platforms the command should be run on. [Valid GOOS and GOARCH values allowed](https://github.com/golang/go/blob/main/src/go/build/syslist.go). Command will be skipped otherwise. |
| `platforms` | `[]string` | All platforms | Specifies which platforms the command should be run on. [Valid GOOS and GOARCH values allowed](https://github.com/golang/go/blob/master/src/go/build/syslist.go). Command will be skipped otherwise. |
| `set` | `[]string` | | Specify options for the [`set` builtin](https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html). |
| `shopt` | `[]string` | | Specify option for the [`shopt` builtin](https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html). |

View File

@@ -5,6 +5,32 @@ sidebar_position: 14
# Changelog
## v3.37.1 - 2024-05-09
- Fix bug where non-string values (numbers, bools) added to `env:` weren't been
correctly exported (#1640, #1641 by @vmaerten and @andreynering).
## v3.37.0 - 2024-05-08
- Released the
[Any Variables experiment](https://taskfile.dev/blog/any-variables), but
[_without support for maps_](https://github.com/go-task/task/issues/1415#issuecomment-2044756925)
(#1415, #1547 by @pd93).
- Refactored how Task reads, parses and merges Taskfiles using a DAG (#1563,
#1607 by @pd93).
- Fix a bug which stopped tasks from using `stdin` as input (#1593, #1623 by
@pd03).
- Fix error when a file or directory in the project contained a special char
like `&`, `(` or `)` (#1551, #1584 by @andreynering).
- Added alias `q` for template function `shellQuote` (#1601, #1603 by @vergenzt)
- Added support for `~` on ZSH completions (#1613 by @jwater7).
- Added the ability to pass variables by reference using Go template syntax when
the
[Map Variables experiment](https://taskfile.dev/experiments/map-variables/) is
enabled (#1612 by @pd93).
- Added support for environment variables in the templating engine in `includes`
(#1610 by @vmaerten).
## v3.36.0 - 2024-04-08
- Added support for

View File

@@ -45,5 +45,5 @@ if you want to adopt the new behavior, you can continue to use the `--force`
flag as you do now!
{/* prettier-ignore-start */}
[enabling-experiments]: /experiments/#enabling-experiments
[enabling-experiments]: ./experiments.mdx#enabling-experiments
{/* prettier-ignore-end */}

View File

@@ -1,11 +1,11 @@
---
slug: /experiments/any-variables/
slug: /experiments/map-variables/
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Any Variables (#1415)
# Map Variables (#1585)
:::caution
@@ -15,19 +15,9 @@ environment. They are intended for testing and feedback only.
:::
Currently, Task only supports string variables. This experiment allows you to
specify and use the following variable types:
- `string`
- `bool`
- `int`
- `float`
- `array`
- `map`
This allows you to have a lot more flexibility in how you use variables in
Task's templating engine. There are two active proposals for this experiment.
Click on the tabs below to switch between them.
Currently, Task supports all variable types except for maps. This experiment
adds two different proposals for map variables. Click on the tabs below to
switch between them.
<Tabs defaultValue="1" queryString="proposal"
values={[
@@ -48,13 +38,11 @@ This experiment proposal breaks the following functionality:
:::info
To enable this experiment, set the environment variable:
`TASK_X_ANY_VARIABLES=1`. Check out [our guide to enabling experiments
`TASK_X_MAP_VARIABLES=1`. Check out [our guide to enabling experiments
][enabling-experiments] for more information.
:::
## Maps
This proposal removes support for the `sh` keyword in favour of a new syntax for
dynamically defined variables, This allows you to define a map directly as you
would for any other type:
@@ -111,19 +99,16 @@ will now need to escape the `$` with a backslash (`\`) to stop Task from
executing it as a command.
</TabItem>
<TabItem value="2">
:::info
To enable this experiment, set the environment variable:
`TASK_X_ANY_VARIABLES=2`. Check out [our guide to enabling experiments
`TASK_X_MAP_VARIABLES=2`. Check out [our guide to enabling experiments
][enabling-experiments] for more information.
:::
## Maps
This proposal maintains backwards-compatibility and the `sh` subkey and adds
another new `map` subkey for defining map variables:
@@ -150,7 +135,13 @@ objects/arrays. This is similar to the `fromJSON` template function, but means
that you only have to parse the JSON/YAML once when you declare the variable,
instead of every time you want to access a value.
Before:
<Tabs defaultValue="2"
values={[
{label: 'Before', value: '1'},
{label: 'After', value: '2'}
]}>
<TabItem value="1">
```yaml
version: 3
@@ -164,7 +155,8 @@ tasks:
- 'echo {{(fromJSON .FOO).b}}'
```
After:
</TabItem>
<TabItem value="2">
```yaml
version: 3
@@ -179,12 +171,26 @@ tasks:
- 'echo {{.FOO.b}}'
```
</TabItem></Tabs>
## Variables by reference
Lastly, this proposal adds support for defining and passing variables by
reference. This is really important now that variables can be types other than a
string. Previously, to send a variable from one task to another, you would have
to use the templating system to pass it:
string.
Previously, to send a variable from one task to another, you would have to use
the templating system. Unfortunately, the templater _always_ outputs a string
and operations on the passed variable may not have behaved as expected. With
this proposal, you can now pass variables by reference using the `ref` subkey:
<Tabs defaultValue="2"
values={[
{label: 'Before', value: '1'},
{label: 'After', value: '2'}
]}>
<TabItem value="1">
```yaml
version: 3
@@ -202,10 +208,8 @@ tasks:
- 'echo {{index .FOO 0}}' # <-- FOO is a string so the task outputs '91' which is the ASCII code for '[' instead of the expected 'A'
```
Unfortunately, this results in the value always being passed as a string as this
is the output type of the templater and operations on the passed variable may
not behave as expected. With this proposal, you can now pass variables by
reference using the `ref` subkey:
</TabItem>
<TabItem value="2">
```yaml
version: 3
@@ -218,12 +222,14 @@ tasks:
- task: bar
vars:
FOO:
ref: FOO # <-- FOO gets passed by reference to bar and maintains its type
ref: .FOO # <-- FOO gets passed by reference to bar and maintains its type
bar:
cmds:
- 'echo {{index .FOO 0}}' # <-- FOO is still a map so the task outputs 'A' as expected
```
</TabItem></Tabs>
This means that the type of the variable is maintained when it is passed to
another Task. This also works the same way when calling `deps` and when defining
a variable and can be used in any combination:
@@ -236,27 +242,54 @@ tasks:
vars:
FOO: [A, B, C] # <-- FOO is defined as an array
BAR:
ref: FOO # <-- BAR is defined as a reference to FOO
ref: .FOO # <-- BAR is defined as a reference to FOO
deps:
- task: bar
vars:
BAR:
ref: BAR # <-- BAR gets passed by reference to bar and maintains its type
ref: .BAR # <-- BAR gets passed by reference to bar and maintains its type
bar:
cmds:
- 'echo {{index .BAR 0}}' # <-- BAR still refers to FOO so the task outputs 'A'
```
All references use the same templating syntax as regular templates, so in
addition to simply calling `.FOO`, you can also pass subkeys (`.FOO.BAR`) or
indexes (`index .FOO 0`) and use functions (`len .FOO`):
```yaml
version: 3
tasks:
foo:
vars:
FOO: [A, B, C] # <-- FOO is defined as an array
cmds:
- task: bar
vars:
FOO:
ref: index .FOO 0 # <-- The element at index 0 is passed by reference to bar
bar:
cmds:
- 'echo {{.MYVAR}}' # <-- FOO is just the letter 'A'
```
</TabItem></Tabs>
---
## Looping over maps
## Common to both proposals
This experiment also adds support for looping over maps using the `for` keyword,
just like arrays. In addition to the `{{.ITEM}}` variable being populated when
looping over a map, we also make an additional `{{.KEY}}` variable available
that holds the string value of the map key.
Both proposals add support for all other variable types by directly defining
them in the Taskfile. For example:
<Tabs defaultValue="1" queryString="proposal"
values={[
{label: 'Proposal 1', value: '1'},
{label: 'Proposal 2', value: '2'}
]}>
### Evaluating booleans
<TabItem value="1">
```yaml
version: 3
@@ -264,64 +297,15 @@ version: 3
tasks:
foo:
vars:
BOOL: false
cmds:
- '{{if .BOOL}}echo foo{{end}}'
```
### Arithmetic
```yaml
version: 3
tasks:
foo:
vars:
INT: 10
FLOAT: 3.14159
cmds:
- 'echo {{add .INT .FLOAT}}'
```
### Ranging
```yaml
version: 3
tasks:
foo:
vars:
ARRAY: [1, 2, 3]
cmds:
- 'echo {{range .ARRAY}}{{.}}{{end}}'
```
There are many more templating functions which can be used with the new types of
variables. For a full list, see the [slim-sprig][slim-sprig] documentation.
## Looping over variables
Previously, you would have to use a delimiter separated string to loop over an
arbitrary list of items in a variable and split them by using the `split` subkey
to specify the delimiter:
```yaml
version: 3
tasks:
foo:
vars:
LIST: 'foo,bar,baz'
MAP: {a: 1, b: 2, c: 3}
cmds:
- for:
var: LIST
split: ','
cmd: echo {{.ITEM}}
var: MAP
cmd: 'echo "{{.KEY}}: {{.ITEM}}"'
```
Both of these proposals add support for looping over "collection-type" variables
using the `for` keyword, so now you are able to loop over a map/array variable
directly:
</TabItem>
<TabItem value="2">
```yaml
version: 3
@@ -329,18 +313,23 @@ version: 3
tasks:
foo:
vars:
LIST: [foo, bar, baz]
map:
MAP: {a: 1, b: 2, c: 3}
cmds:
- for:
var: LIST
cmd: echo {{.ITEM}}
var: MAP
cmd: 'echo "{{.KEY}}: {{.ITEM}}"'
```
When looping over a map we also make an additional `{{.KEY}}` variable availabe
that holds the string value of the map key. Remember that maps are unordered, so
:::note
Remember that maps are unordered, so
the order in which the items are looped over is random.
:::
</TabItem></Tabs>
{/* prettier-ignore-start */}
[enabling-experiments]: /experiments/#enabling-experiments
[slim-sprig]: https://go-task.github.io/slim-sprig/
[enabling-experiments]: ./experiments.mdx#enabling-experiments
{/* prettier-ignore-end */}

View File

@@ -48,6 +48,20 @@ tasks:
and you run `task my-remote-namespace:hello`, it will print the text: "Hello
from the remote Taskfile!" to your console.
The Taskfile location is processed by the templating system, so you can
reference environment variables in your URL if you need to add authentication.
For example:
```yaml
version: '3'
includes:
my-remote-namespace: https://{{.TOKEN}}@raw.githubusercontent.com/my-org/my-repo/main/Taskfile.yml
```
`TOKEN=my-token task my-remote-namespace:hello` will be resolved by Task to
`https://my-token@raw.githubusercontent.com/my-org/my-repo/main/Taskfile.yml`
## Security
Running commands from sources that you do not control is always a potential
@@ -99,6 +113,6 @@ the `--timeout` flag and specifying a duration. For example, `--timeout 5s` will
set the timeout to 5 seconds.
{/* prettier-ignore-start */}
[enabling-experiments]: /experiments/#enabling-experiments
[enabling-experiments]: ./experiments.mdx#enabling-experiments
[man-in-the-middle-attacks]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
{/* prettier-ignore-end */}

View File

@@ -38,5 +38,5 @@ information.
\{Short explanation of how users should migrate to the new behavior\}
{/* prettier-ignore-start */}
[enabling-experiments]: /experiments/#enabling-experiments
[enabling-experiments]: ./experiments.mdx#enabling-experiments
{/* prettier-ignore-end */}

View File

@@ -31,7 +31,7 @@ brew install go-task
### pkgx
If you're on macOS or Linux and have [pkgx](https://pkgx.sh/) installed, getting Task is as
If you're on macOS or Linux and have [pkgx][pkgx] installed, getting Task is as
simple as running:
```shell
@@ -299,5 +299,5 @@ Invoke-Expression -Command path/to/task.ps1
[godownloader]: https://github.com/goreleaser/godownloader
[choco]: https://chocolatey.org/
[scoop]: https://scoop.sh/
[tea]: https://tea.xyz/
[pkgx]: https://pkgx.sh/
{/* prettier-ignore-end */}

View File

@@ -256,8 +256,8 @@ The variable priority order was also different:
4. `Taskvars.yml` variables
{/* prettier-ignore-start */}
[deprecate-version-2-schema]: /deprecations/version-2-schema/
[output]: /usage#output-syntax
[ignore_errors]: /usage#ignore-errors
[includes]: /usage#including-other-taskfiles
[deprecate-version-2-schema]: ./deprecations/version_2_schema.mdx
[output]: ./usage.mdx#output-syntax
[ignore_errors]: ./usage.mdx#ignore-errors
[includes]: ./usage.mdx#including-other-taskfiles
{/* prettier-ignore-end */}

View File

@@ -121,13 +121,14 @@ tasks:
### Reading a Taskfile from stdin
Taskfile also supports reading from stdin. This is useful if you are generating
Taskfiles dynamically and don't want write them to disk. This works just like
any other program that supports stdin. For example:
Taskfiles dynamically and don't want write them to disk. To tell task to read
from stdin, you must specify the `-t/--taskfile` flag with the special `-`
value. You may then pipe into Task as you would any other program:
```shell
task < <(cat ./Taskfile.yml)
task -t - <(cat ./Taskfile.yml)
# OR
cat ./Taskfile.yml | task
cat ./Taskfile.yml | task -t -
```
## Environment variables
@@ -947,8 +948,26 @@ tasks:
## Variables
When doing interpolation of variables, Task will look for the below. They are
listed below in order of importance (i.e. most important first):
Task allows you to set variables using the `vars` keyword. The following
variable types are supported:
- `string`
- `bool`
- `int`
- `float`
- `array`
:::note
Maps are not supported by default, but there is an
[experiment][map-variables] that can be enabled to add support. If
you're interested in this functionality, we would appreciate your feedback.
:pray:
:::
Variables can be set in many places in a Taskfile. When executing templates,
Task will look for variables in the order listed below (most important first):
- Variables declared in the task definition
- Variables given while calling a task from another (See
@@ -1093,8 +1112,8 @@ tasks:
### Looping over variables
To loop over the contents of a variable, you simply need to specify the variable
you want to loop over. By default, variables will be split on any whitespace
characters.
you want to loop over. By default, string variables will be split on any
whitespace characters.
```yaml
version: '3'
@@ -1108,8 +1127,8 @@ tasks:
cmd: cat {{.ITEM}}
```
If you need to split on a different character, you can do this by specifying the
`split` property:
If you need to split a string on a different character, you can do this by
specifying the `split` property:
```yaml
version: '3'
@@ -1123,6 +1142,26 @@ tasks:
cmd: cat {{.ITEM}}
```
You can also loop over arrays directly (and maps if you have the
[maps experiment](/experiments/map-variables) enabled):
```yaml
version: 3
tasks:
foo:
vars:
LIST: [foo, bar, baz]
cmds:
- for:
var: LIST
cmd: echo {{.ITEM}}
```
When looping over a map we also make an additional `{{.KEY}}` variable available
that holds the string value of the map key. Remember that maps are unordered, so
the order in which the items are looped over is random.
All of this also works with dynamic variables!
```yaml
@@ -1377,7 +1416,7 @@ Task also adds the following functions:
converts a string from `/` path format to `\`.
- `exeExt`: Returns the right executable extension for the current OS (`".exe"`
for Windows, `""` for others).
- `shellQuote`: Quotes a string to make it safe for use in shell scripts. Task
- `shellQuote` (aliased to `q`): Quotes a string to make it safe for use in shell scripts. Task
uses [this Go function](https://pkg.go.dev/mvdan.cc/sh/v3@v3.4.0/syntax#Quote)
for this. The Bash dialect is assumed.
- `splitArgs`: Splits a string as if it were a command's arguments. Task uses
@@ -1956,4 +1995,5 @@ if called by another task, either directly or as a dependency.
{/* prettier-ignore-start */}
[gotemplate]: https://golang.org/pkg/text/template/
[map-variables]: ./experiments/map_variables.mdx
{/* prettier-ignore-end */}