feat(cli): persist env, global-env, and collection var changes from scripts (#8387)

* feat(variables): implement variable persistence and updates for environment and collection files

- Introduced new utility functions for applying and persisting variable updates, enhancing the management of environment and collection variables.
- Updated the run command to include descriptors for environment files, allowing for better tracking of variable changes.
- Enhanced the runSingleRequest function to synchronize variable updates and persist changes to the appropriate files.
- Added comprehensive tests to ensure the correct functionality of variable merging and persistence logic across different file formats.

* fix(variables): refine YAML handling and data type inference for environment variables

- Updated the run command to exclusively recognize '.yml' files, removing support for '.yaml' extensions.
- Enhanced the persistence logic to correctly handle data type inference for newly added environment and collection variables.
- Added comprehensive tests to validate the correct writing of variables to YAML and .bru files, ensuring accurate data type preservation and updates.
- Improved the handling of existing data types when variables are modified, ensuring consistency across different formats.

* feat(tests): add integration tests for typed variable persistence in CLI

- Introduced a new test suite to validate the persistence of typed environment and collection variables set via scripts in the CLI.
- Implemented a local server to facilitate HTTP requests during tests, ensuring accurate simulation of variable management.
- Enhanced the test logic to verify that the correct data type annotations are written to both .bru and YAML files after CLI execution.
- Added comprehensive assertions to confirm the expected disk state for both global and collection variables across different sandboxes.

* refactor(runSingleRequest): remove synchronization of variable updates after post response script execution

- Eliminated the syncVariableUpdates call from the runSingleRequest function, streamlining the variable handling process.
- Adjusted the logic to focus on executing post response variables without unnecessary synchronization, improving performance and clarity.

* feat(env-vars): implement transient environment variable handling and improve persistence resilience

- Added support for transient environment variable overrides via the `--env-var` CLI option, ensuring these values are not persisted to disk.
- Enhanced the `mergeScriptVarsIntoEnvList` function to prevent overridden variables from being written back, preserving existing entries.
- Updated the `persistVariableUpdates` function to handle potential disk write errors gracefully, logging warnings instead of failing the execution.
- Introduced comprehensive tests to validate the behavior of transient variables and ensure correct persistence logic under various scenarios.

* fix(persist-variables): enhance variable persistence logic and testing

- Improved the `persistEnvFile` function to preserve additional fields (uid, dataType, custom metadata) during JSON writes, ensuring no data loss for unrecognized entries.
- Updated the `persistVariableUpdates` function to respect `envVarOverrides` when writing to the global environment file, preventing transient values from being persisted.
- Added regression tests to validate the preservation of additional fields and the correct handling of environment variable overrides in persistence scenarios.

* refactor(tests): streamline fixture writing and enhance variable persistence tests

- Renamed the `writeFileSyncMkdirP` function to `writeFixtureFile` for clarity, emphasizing its role in writing fixture files with parent directory creation.
- Replaced multiple calls to `writeFileSyncMkdirP` with the new `writeFixtureFile` function to improve code readability and maintainability.
- Added tests to verify the deletion of environment variables from disk when removed from the environment map, ensuring accurate persistence behavior.
- Implemented checks to prevent runtime variables from being persisted, safeguarding against unintended data leaks.

* test(env-file): add integration test for JSON environment variable persistence

- Implemented a new test to verify that typed environment variables are correctly persisted to a JSON file using the --env-file option.
- Ensured that existing entries retain their uid and custom fields during the persistence process, validating the shape-preservation guarantee.
- Enhanced the test to check that new typed variables from the script are accurately written and that untouched entries remain unchanged.

* test(env-file): add integration tests for YAML and .bru variable persistence

- Implemented new tests to verify the persistence of typed environment variables to both YAML and .bru files using the --env-file option.
- Ensured that typed values are correctly serialized with appropriate annotations in the output files, validating the correct handling of different formats.
- Enhanced coverage for the persistence behavior of environment variables, confirming that existing entries remain unchanged while new variables are accurately written.

* test(env-file): enhance JSON environment variable persistence tests

- Added a new test to verify the preservation of native typed values in JSON environment files when unrelated keys are touched during script execution.
- Ensured that typed values maintain their original types and are auto-annotated with the correct dataType, validating the integrity of the environment variable persistence process.
- Expanded the typed-value inference tests to cover cross-type transitions, confirming that existing dataType annotations are ignored when new values are written.

* refactor(env-vars): transition env-var overrides to Map for improved persistence handling

- Changed the `envVarOverrides` from a Set to a Map to track injected values, enhancing the logic for distinguishing between transient and deliberate variable writes.
- Updated related functions to accommodate the new Map structure, ensuring that only matching overrides are filtered out during persistence.
- Enhanced documentation and tests to reflect the new behavior, confirming that deliberate script writes with different values are correctly persisted.

* test(integration): refine comments and enhance clarity in typed persistence tests

* test(integration): add tests for variable persistence in tests blocks and error handling

- Implemented new tests to verify that variables set within `tests {}` blocks are correctly persisted to the environment and collection files.
- Added a test to ensure that variables written before an error in a `tests {}` block are still saved, confirming the integrity of partial results during execution.
- Enhanced existing tests with detailed comments for clarity and understanding of the persistence behavior in various scenarios.

* test(integration): add collection variable persistence test for error handling

- Introduced a new test to verify that collection variables set before an error in a `tests {}` block are correctly persisted to the collection file.
- Enhanced existing tests to ensure that both environment and collection variables maintain their expected values during error scenarios, confirming the robustness of variable persistence.

* test(integration): add test for environment and collection variable persistence from post-response expressions

- Introduced a new test to verify that environment and collection variables mutated as a side effect of `vars:post-response` expressions are correctly persisted.
- Enhanced the `runSingleRequest` function to synchronize variable updates after executing post-response scripts, ensuring that changes are reflected in the environment and collection files.
- This change improves the robustness of variable handling in the CLI, aligning it with the expected behavior of the desktop application.

* refactor(persist-variables): streamline JSON parsing and enhance variable persistence tests

- Simplified the JSON parsing logic in the `persistEnvFile` function by removing unnecessary try-catch blocks, ensuring that malformed files are handled gracefully.
- Updated tests to verify the persistence of new collection variable types, including numbers, booleans, and objects, ensuring that typed variables are correctly serialized and maintained.
- Enhanced existing tests to confirm that plain string variables remain unchanged during persistence, improving the robustness of variable handling in the CLI.

* jsdoc change

packages/bruno-cli/src/runner/run-single-request.js

@@ -9,6 +9,7 @@ const { interpolateString, interpolateObject } = require('./interpolate-string')
 const { ScriptRuntime, TestRuntime, VarsRuntime, AssertRuntime, formatErrorWithContext, SCRIPT_TYPES } = require('@usebruno/js');
 const { stripExtension } = require('../utils/filesystem');
 const { getOptions } = require('../utils/bru');
+const { applyVariableUpdates, persistVariableUpdates } = require('../utils/persist-variables');
 const { makeAxiosInstance } = require('../utils/axios-instance');
 const { addAwsV4Interceptor, resolveAwsV4Credentials } = require('./awsv4auth-helper');
 const { setupProxyAgents } = require('../utils/proxy-util');
@@ -93,8 +94,31 @@ const runSingleRequest = async function (
   runtime,
   collection,
   runSingleRequestByPathname,
-  globalEnvVars = {}
+  globalEnvVars = {},
+  persistPaths = {}
 ) {
+  const syncVariableUpdates = (result, currentRequest) => {
+    if (!result) return;
+    applyVariableUpdates(result, {
+      envVariables,
+      runtimeVariables,
+      globalEnvVars,
+      request: currentRequest
+    });
+    // Persistence is a side effect — never tank the run for it. In CI, env files may sit on
+    // read-only mounts or the user's shell may lack write permissions; log and continue.
+    try {
+      persistVariableUpdates(result, {
+        envFile: persistPaths.envFile,
+        globalEnvFile: persistPaths.globalEnvFile,
+        collection,
+        collectionRootPath: persistPaths.collectionRootPath,
+        envVarOverrides: persistPaths.envVarOverrides
+      });
+    } catch (err) {
+      console.warn(chalk.yellow(`Warning: failed to persist variable updates: ${err.message}`));
+    }
+  };
   const { pathname: itemPathname } = item;
   const relativeItemPathname = path.relative(collectionPath, itemPathname);
 
@@ -228,6 +252,7 @@ const runSingleRequest = async function (
           scriptingConfig,
           runSingleRequestByPathname,
           collectionName);
+        syncVariableUpdates(result, request);
         if (result?.nextRequestName !== undefined) {
           nextRequestName = result.nextRequestName;
         }
@@ -281,6 +306,9 @@ const runSingleRequest = async function (
         // Extract partial results from the error (tests that passed before the error)
         preRequestTestResults = error?.partialResults?.results || [];
 
+        // Persist any variable changes the script made before erroring
+        syncVariableUpdates(error?.partialResults, request);
+
         // Preserve nextRequestName if it was set before the error
         if (error?.partialResults?.nextRequestName !== undefined) {
           nextRequestName = error.partialResults.nextRequestName;
@@ -742,7 +770,7 @@ const runSingleRequest = async function (
     const postResponseVars = get(item, 'request.vars.res');
     if (postResponseVars?.length) {
       const varsRuntime = new VarsRuntime({ runtime: scriptingConfig?.runtime });
-      varsRuntime.runPostResponseVars(
+      const result = varsRuntime.runPostResponseVars(
         postResponseVars,
         request,
         response,
@@ -751,6 +779,9 @@ const runSingleRequest = async function (
         collectionPath,
         processEnvVars
       );
+      // Expressions can invoke bru.setEnvVar / setGlobalEnvVar / setCollectionVar as a side effect,
+      // mirroring how the desktop app surfaces these mutations after the vars block.
+      syncVariableUpdates(result, request);
     }
 
     // run post response script
@@ -771,6 +802,7 @@ const runSingleRequest = async function (
           runSingleRequestByPathname,
           collectionName
         );
+        syncVariableUpdates(result, request);
         if (result?.nextRequestName !== undefined) {
           nextRequestName = result.nextRequestName;
         }
@@ -802,6 +834,8 @@ const runSingleRequest = async function (
           }
         ];
 
+        syncVariableUpdates(error?.partialResults, request);
+
         if (error?.partialResults?.nextRequestName !== undefined) {
           nextRequestName = error.partialResults.nextRequestName;
         }
@@ -847,6 +881,7 @@ const runSingleRequest = async function (
           runSingleRequestByPathname,
           collectionName
         );
+        syncVariableUpdates(result, request);
         testResults = get(result, 'results', []);
 
         if (result?.nextRequestName !== undefined) {
@@ -879,6 +914,8 @@ const runSingleRequest = async function (
           }
         ];
 
+        syncVariableUpdates(error?.partialResults, request);
+
         if (error?.partialResults?.nextRequestName !== undefined) {
           nextRequestName = error.partialResults.nextRequestName;
         }