Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Nx launches only 1st target of dependsOn when using custom plugin in v18-v19 #23496

Open
1 of 4 tasks
khludenevav opened this issue May 17, 2024 · 6 comments
Open
1 of 4 tasks

Nx launches only 1st target of dependsOn when using custom plugin in v18-v19 #23496

khludenevav opened this issue May 17, 2024 · 6 comments
Assignees
@xiongemi
Labels
blocked: more info needed scope: core core nx functionality type: bug

Comments

@khludenevav
Copy link

khludenevav commented May 17, 2024 •
edited

Current Behavior

I'm trying to migrate 18.0.5 to 19.0.4:
I tried latest v18 2 days ago and it doesn't work.
I tried v18.1.2 19 Mar and it also doesn't work.

I have such project.json config in my docs project.

{
  "name": "docs",
  "$schema": "../../node_modules/nx/schemas/project-schema.json",
  "sourceRoot": "apps/docs",
  "projectType": "application",
  "implicitDependencies": ["nx-plugin", "styles"],
  "targets": {
    "generate-open-api-registry": {
      "executor": "nx-plugin:generate-open-api-registry",
      "outputs": ["{options.outputOpenApiRegistryPath}"],
      "options": {
        "openApiDefinitionV1Path": "../developer_portal/openapi/file_v1.yaml",
        "openApiDefinitionV2Path": "../developer_portal/openapi/file_v2.yaml",
        "outputOpenApiRegistryPath": "{projectRoot}/generated/open-api-registry.json"
      }
    },
    "generate-changelog-registry": {
      "executor": "nx-plugin:generate-changelog-registry",
      "outputs": ["{options.outputChangelogRegistry}"],
      "options": {
        "repoRoot": "../",
        "changelogRoot": "../changelog",
        "outputChangelogRegistry": "{projectRoot}/generated/changelog-registry.json"
      }
    },
    "preprocess-docs": {
      "executor": "nx:run-commands",
      "dependsOn": ["generate-open-api-registry", "generate-changelog-registry"],
      "options": {
        "commands": ["echo preprocess-docs target successful"]
      }
    },

    "test1": {
      "executor": "nx:run-commands",
      "options": {
        "commands": ["echo test1"]
      }
    },
    "test2": {
      "executor": "nx:run-commands",
      "options": {
        "commands": ["echo test2"]
      }
    },
    "test3": {
      "executor": "nx:run-commands",
      "options": {
        "commands": ["echo test3"]
      }
    },
    "test4": {
      "dependsOn": ["test1", "test2", "test3"],
      "executor": "nx:run-commands",
      "options": {
        "commands": ["echo test4"]
      }
    }
  },
  "tags": []
}

Generator code in attachments.

When I launch on CI (we use buildkite) (locally work perfectly)
pnpm exec nx run docs:preprocess-docs --verbose
I see nx output which says that nx launches first dependency of preprocess-docs (generate-open-api-registry), but doesn't launch 2nd. If I add another dependency to preprocess-docs, seems only 1st one launched.


NX   Running target preprocess-docs for project docs and 2 tasks it depends on:
--
  |  
  |  
  |  
  | > nx run docs:generate-open-api-registry
  |  
  |  
  | > nx run docs:generate-changelog-registry
  |  
  | generate-open-api-registry launched
  | generate-open-api-registry successfully executed
  |  
  |  
  |  
  | NX   Running target preprocess-docs for project docs and 2 tasks it depends on failed
  |  
  | Failed tasks:
  |  
  | - docs:generate-changelog-registry
  |  
  | Hint: run the command with --verbose for more details.
  |  
  | 🚨 Error: The command exited with status 1

Probably it is related to my custom nx-plugin. Because
pnpm exec nx run docs:test4 --verbose
task seems (according on output) launched normally.

NX   Running target test4 for project docs and 3 tasks it depends on:
--
  |  
  |  
  |  
  | > nx run docs:test1
  |  
  |  
  | > nx run docs:test2
  |  
  |  
  | > nx run docs:test3
  |  
  | > echo test1
  |  
  | > echo test2
  |  
  | > echo test3
  |  
  | test1
  | test2
  | test3
  |  
  | > nx run docs:test4
  |  
  | > echo test4
  |  
  | test4
  |  
  |  
  |  
  | NX   Successfully ran target test4 for project docs and 3 tasks it depends on

Expected Behavior

Task preprocess-docs and its deps passed.

GitHub Repo

No response

Steps to Reproduce

^^

Nx Report

NX   Report complete - copy this into the issue template

Node   : 20.10.0
OS     : darwin-arm64
pnpm   : 9.1.0

nx (global)        : 19.0.4
nx                 : 19.0.4
@nx/js             : 19.0.4
@nx/jest           : 19.0.4
@nx/eslint         : 19.0.4
@nx/workspace      : 19.0.4
@nx/devkit         : 19.0.4
@nx/eslint-plugin  : 19.0.4
@nx/next           : 19.0.4
@nx/plugin         : 19.0.4
@nx/react          : 19.0.4
@nx/storybook      : 19.0.4
@nx/web            : 19.0.4
typescript         : 5.4.5
---------------------------------------
Registered Plugins:
nx-plugin

Failure Logs

No response

Package Manager Version

No response

Operating System

  • macOS
  • Linux
  • Windows
  • Other (Please specify)

Additional Information

Archive.zip

No response
@khludenevav khludenevav changed the title dependsOn works incorrectly on CI after 18.0.5 when using custom plugin Nx launches only 1st target of dependsOn when using custom plugin in v18-v19 May 17, 2024
@FrozenPandaz
Copy link
Collaborator

Could you provide more information about your CI pipeline please? Are you using Nx Cloud? Are there any environment variables or options set? If possible could you get the latest version that did work for you?

@khludenevav khludenevav mentioned this issue May 17, 2024
Open
4 tasks
@khludenevav
Copy link
Author

khludenevav commented May 17, 2024 •
edited

I'm using https://buildkite.com instead of Nx Cloud.
Good thing here is I can reproduce stably that issue.

Are there any environment variables or options set?

Seems no options. It is just call of executor of local nx-plugin.

Env vars 
  {
  "SHELL": "/bin/sh",
  "BUILDKITE_GIT_MIRRORS_LOCK_TIMEOUT": "300",
  "BUILDKITE_AGENT_META_DATA_QUEUE": "spot-medium",
  "BUILDKITE_TAG": "",
  "BUILDKITE_GIT_CHECKOUT_FLAGS": "-f",
  "BUILDKITE_BUILD_ID": "018f871c-163d-4861-868f-5b92b16e3fe8",
  "BUILDKITE_REBUILT_FROM_BUILD_ID": "",
  "NVM_INC": "/var/lib/buildkite-agent/.nvm/versions/node/v20.10.0/include/node",
  "BUILDKITE_GIT_FETCH_FLAGS": "-v --prune",
  "BUILDKITE_STEP_IDENTIFIER": "generate-og-images",
  "BUILDKITE_BUILD_NUMBER": "184704",
  "BUILDKITE_SHELL": "/bin/bash -e -c",
  "BUILDKITE_PIPELINE_NAME": "engineering-tests",
  "BUILDKITE_STRICT_SINGLE_HOOKS": "false",
  "BUILDKITE": "true",
  "BUILDKITE_LAST_HOOK_EXIT_STATUS": "0",
  "BUILDKITE_STEP_KEY": "generate-og-images",
  "BUILDKITE_GIT_SUBMODULES": "true",
  "BUILDKITE_GIT_MIRRORS_PATH": "",
  "BUILDKITE_SCRIPT_PATH": "cd .buildkite/scripts/sync-og-images\nbash ./update-og-images.sh\n",
  "BUILDKITE_PULL_REQUEST_BASE_BRANCH": "main",
  "BUILDKITE_AGENT_DISABLE_WARNINGS_FOR": "",
  "BUILDKITE_AGENT_JWKS_KEY_ID": "BUILDKITE_PIPELINE_KEY_A",
  "PWD": "/home/buildkite/root/client",
  "BUILDKITE_ENV_FILE": "/tmp/job-env-018f871c-8060-4010-80b9-32e028a2ec7d2123230743",
  "BUILDKITE_BIN_PATH": "/usr/bin",
  "LOGNAME": "buildkite-agent",
  "BUILDKITE_PLUGINS_PATH": "/etc/buildkite-agent/plugins",
  "BUILDKITE_ARTIFACT_PATHS": "",
  "BUILDKITE_CONFIG_PATH": "/etc/buildkite-agent/buildkite-agent.cfg",
  "COMPOSE_HTTP_TIMEOUT": "180",
  "SYSTEMD_EXEC_PID": "1715",
  "BUILDKITE_GIT_CLONE_FLAGS": "-v",
  "BUILDKITE_GIT_CLEAN_FLAGS": "-ffxdq",
  "BUILDKITE_AGENT_NAME": "buildkite-agent-kn2k-1",
  "BUILDKITE_COMMAND_EVAL": "true",
  "NX_REMOTE_CACHE_BUCKET": "nx-cache",
  "BUILDKITE_LABEL": "generate-og-images",
  "BUILDKITE_BUILD_CHECKOUT_PATH": "/home/buildkite/root",
  "BUILDKITE_BUILD_CREATOR": "khludenevav",
  "HOME": "/var/lib/buildkite-agent",
  "LANG": "C.UTF-8",
  "BUILDKITE_AGENT_ENDPOINT": "https://agent.buildkite.com/v3",
  "BUILDKITE_GCS_ACCESS_HOST": "storage.cloud.google.com",
  "CLOUDSDK_CONFIG": "/tmp/.config/gcloud",
  "BUILDKITE_SOCKETS_PATH": "/var/lib/buildkite-agent/.buildkite-agent/sockets",
  "BUILDKITE_AGENT_PID": "1715",
  "BUILDKITE_PIPELINE_PROVIDER": "github",
  "BUILDKITE_REBUILT_FROM_BUILD_NUMBER": "",
  "BUILDKITE_AGENT_JWKS_FILE": "/home/buildkite/buildkite_pipeline_signing_a",
  "BUILDKITE_REPO_SSH_HOST": "github.com",
  "INVOCATION_ID": "2fa38c9f5be74a2a81b6cdb5c1ddad66",
  "BUILDKITE_COMPUTE_TYPE": "self-hosted",
  "BUILDKITE_COMMAND": "cd .buildkite/scripts/sync-og-images\nbash ./update-og-images.sh\n",
  "BUILDKITE_HOOKS_PATH": "/etc/buildkite-agent/hooks",
  "BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG": "",
  "BUILDKITE_BUILD_AUTHOR": "khludenevav",
  "BUILDKITE_PULL_REQUEST": "177668",
  "DOCKERHUB_PASSWORD": "[REDACTED]",
  "NVM_DIR": "/var/lib/buildkite-agent/.nvm",
  "BUILDKITE_PULL_REQUEST_DRAFT": "true",
  "BUILDKITE_JOB_ID": "018f871c-8060-4010-80b9-32e028a2ec7d",
  "TERM": "xterm-256color",
  "BUILDKITE_AGENT_ID": "018f8718-d39a-4e6f-8d26-16298be1b22d",
  "USER": "buildkite-agent",
  "BUILDKITE_PLUGINS_ENABLED": "true",
  "BUILDKITE_STEP_ID": "018f871c-7fbb-4ac6-bb60-b5e0fcd34f10",
  "BUILDKITE_GIT_MIRRORS_SKIP_UPDATE": "false",
  "BUILDKITE_AGENT_DEBUG": "false",
  "BUILDKITE_BRANCH": "khludenev/fix-og-images",
  "BUILDKITE_PIPELINE_DEFAULT_BRANCH": "",
  "BUILDKITE_PROJECT_PROVIDER": "github",
  "SHLVL": "1",
  "NVM_CD_FLAGS": "",
  "DOCKERHUB_LOGIN": "it5t",
  "BUILDKITE_BUILD_PATH": "/var/lib/buildkite-agent/builds",
  "BUILDKITE_SOURCE": "webhook",
  "BUILDKITE_AGENT_EXPERIMENT": "resolve-commit-after-checkout",
  "FLAKY_TEST_ATTEMPTS": "",
  "BUILDKITE_LOCAL_HOOKS_ENABLED": "true",
  "BUILDKITE_AGENT_JOB_API_TOKEN": "[REDACTED]",
  "BUILDKITE_SSH_KEYSCAN": "true",
  "BUILDKITE_SIGNAL_GRACE_PERIOD_SECONDS": "9",
  "BUILDKITE_PIPELINE_ID": "0188d2ed-1f02-41a7-b07f-3291eef8a453",
  "BUILDKITE_TIMEOUT": "60",
  "BUILDKITE_AGENT_DEBUG_HTTP": "false",
  "BUILDKITE_PIPELINE_SLUG": "engineering-tests",
  "BUILDKITE_AGENT_ACCESS_TOKEN": "[REDACTED]",
  "SPOT_QUEUE": "spot-",
  "JOURNAL_STREAM": "8:23728",
  "BUILDKITE_MESSAGE": "get env vars",
  "BUILDKITE_TRIGGERED_FROM_BUILD_NUMBER": "",
  "PATH": "/var/lib/buildkite-agent/.nvm/versions/node/v20.10.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/usr/bin:/home/buildkite/jdk/zulu17.46.19-ca-jdk17.0.9-linux_x64/bin:/opt/poetry/bin",
  "BUILDKITE_PIPELINE_TEAMS": "everyone",
  "CI": "true",
  "BUILDKITE_GIT_CLONE_MIRROR_FLAGS": "-v",
  "BUILDKITE_PLUGIN_VALIDATION": "false",
  "NVM_BIN": "/var/lib/buildkite-agent/.nvm/versions/node/v20.10.0/bin",
  "DOCKER_CLIENT_TIMEOUT": "180",
  "BUILDKITE_RETRY_COUNT": "0",
  "BUILDKITE_REDACTED_VARS": "*_PASSWORD,*_SECRET,*_TOKEN,*_PRIVATE_KEY,*_ACCESS_KEY,*_SECRET_KEY,*_CONNECTION_STRING",
  "GIT_TERMINAL_PROMPT": "0",
  "BUILDKITE_COMMIT": "e228c7c14fb4d09c7b063b2d2c88b3748576efc9",
  "BUILDKITE_TRIGGERED_FROM_BUILD_ID": "",
  "BUILDKITE_BUILD_CREATOR_TEAMS": "",
  "OLDPWD": "/home/buildkite/root/.buildkite/scripts/sync-og-images",
  "_": "/var/lib/buildkite-agent/.nvm/versions/node/v20.10.0/bin/node"
}

If possible could you get the latest version that did work for you?

Sorry, I spend already too mich time to rollout and rollback.

@github-actions GitHub Actions
Copy link

This issue has been automatically marked as stale because more information has not been provided within 7 days.
It will be closed in 21 days if no information is provided.
If information has been provided, please reply to keep it active.
Thanks for being a part of the Nx community! 🙏

@github-actions github-actions bot added the stale label May 25, 2024
@khludenevav
Copy link
Author

khludenevav commented May 25, 2024 •
edited

This issue is significant for my company! Our build is unstable because we can't upgrade and solve that issue (I hope the upgrade helps 😔...)

Also, 21 days have not passed.
image

@github-actions github-actions bot removed the stale label May 26, 2024
@khludenevav
Copy link
Author

khludenevav commented May 26, 2024 •
edited

Sorry for annoying, but just wanted to say it is very urgent for me. Pipeline fails every day at each 2nd PR on nextjs build. We forced to press rebuild manually. It makes angry around 400 employees.

I investigated and for me seems it broken in the release https://github.com/nrwl/nx/releases/tag/18.1.0-beta.10

Probably core: run commands directly (#21918) feature

I'm completely ok if you give me a version number (like 19.2.1) and pnpm patch file, so I can test the fix on CI.

package.json content:

"pnpm": {
    "patchedDependencies": {
      "next@13.4.5": "patches/next@13.4.5.patch",
    }
  },
Checked versions list.

It is output of npm view nx time --json with marked worked and do not worked versions

  "18.0.4": "2024-02-09T23:42:31.904Z",
  "18.1.0-canary.20240210-b1d0294": "2024-02-10T03:14:35.411Z",
  "18.1.0-beta.0": "2024-02-10T20:52:04.366Z",
  "18.1.0-canary.20240213-7d2cb37": "2024-02-13T03:18:17.022Z",
  "18.1.0-canary.20240214-b625a79": "2024-02-14T03:16:46.207Z",
WORKS  "18.1.0-canary.20240215-d5e1451": "2024-02-15T03:19:17.783Z",
  "18.1.0-canary.20240216-27cf308": "2024-02-16T03:18:34.971Z",
  "18.1.0-canary.20240217-5b6d1b4": "2024-02-17T03:14:31.551Z",
  "18.1.0-canary.20240220-003b961": "2024-02-20T03:15:53.478Z",
  "18.1.0-beta.1": "2024-02-20T19:01:40.505Z",
  "18.1.0-beta.2": "2024-02-20T21:08:46.427Z",
  "18.1.0-canary.20240221-abc0cf8": "2024-02-21T03:14:38.982Z",
  "18.1.0-canary.20240222-5d6abe4": "2024-02-22T03:14:35.256Z",
  "18.1.0-canary.20240223-21e7648": "2024-02-23T03:18:59.825Z",
  WORKS (my current version) “18.0.5": "2024-02-24T01:55:51.594Z",
  "18.1.0-canary.20240224-26b266f": "2024-02-24T03:18:35.691Z",
  "18.1.0-canary.20240227-452d845": "2024-02-27T03:21:05.133Z",
  "18.1.0-canary.20240228-4d13753": "2024-02-28T03:19:19.052Z",
  "18.1.0-canary.20240229-f6a183c": "2024-02-29T03:19:39.642Z",
  "18.0.6": "2024-02-29T23:13:52.901Z",
  "18.1.0-canary.20240301-d705372": "2024-03-01T03:16:49.999Z",
  "18.1.0-canary.20240302-8bde48f": "2024-03-02T03:21:19.579Z",
BROKEN STUCK on pnpm i  "18.1.0": "2024-03-04T09:27:11.565Z",
  "18.0.7": "2024-03-04T14:55:05.287Z",
  "18.1.0-beta.3": "2024-03-04T15:55:45.389Z",
  "18.1.0-canary.20240306-cc2f655": "2024-03-06T03:22:58.456Z",
 BROKEN STUCK on pnpm i "18.1.0-beta.4": "2024-03-06T18:03:22.682Z",
  "18.1.0-beta.5": "2024-03-06T23:41:00.578Z",
  "18.1.0-canary.20240307-84d96cc": "2024-03-07T03:16:13.020Z",
  "18.1.0-canary.20240308-46592d7": "2024-03-08T03:16:17.726Z",
  "18.1.0-beta.6": "2024-03-08T15:55:14.598Z",
WORKS "18.0.8": "2024-03-08T20:50:31.855Z",
  "18.1.0-canary.20240309-235ca8c": "2024-03-09T03:14:41.823Z",
  "18.1.0-beta.7": "2024-03-09T04:00:22.005Z",
  "18.1.0-beta.8": "2024-03-09T16:59:34.006Z",
WORKS  "18.1.0-beta.9": "2024-03-11T20:06:05.817Z",
  "18.1.0-canary.20240312-9520aa2": "2024-03-12T03:15:24.771Z",
  "18.1.0-canary.20240313-81df848": "2024-03-13T03:15:41.246Z",
  WORKS 18.1.0-canary.20240314-cbb88f0": "2024-03-14T03:14:09.284Z",
  WORKS 18.1.0-canary.20240315-3b8cbae": "2024-03-15T03:14:31.104Z",
 BROKEN "18.1.0-beta.10": "2024-03-15T14:36:06.061Z",
 BROKEN “18.1.1": "2024-03-15T21:08:22.068Z",
  "18.2.0-canary.20240316-b69047e": "2024-03-16T03:16:18.447Z",
  "18.1.2": "2024-03-18T14:56:05.537Z",
  "18.2.0-canary.20240319-55f31cf": "2024-03-19T03:20:54.822Z",
  "18.2.0-canary.20240320-64b2396": "2024-03-20T03:16:05.949Z",
  "18.2.0-canary.20240321-2a4c57d": "2024-03-21T03:14:34.722Z",
  "18.2.0-canary.20240322-61cb63d": "2024-03-22T03:19:22.695Z",
  "18.2.0-beta.0": "2024-03-22T19:01:08.631Z",
  "18.2.0-canary.20240323-54d4780": "2024-03-23T03:18:10.543Z",
BROKEN  "18.1.3": "2024-03-25T16:26:12.030Z",
  "18.2.0-beta.1": "2024-03-25T17:57:39.791Z",
  "18.2.0-canary.20240326-fb90767": "2024-03-26T03:19:58.803Z",
  "18.2.0-canary.20240327-82dc703": "2024-03-27T03:19:10.311Z",
  "18.2.0-beta.2": "2024-03-27T17:39:47.039Z",
  "18.2.0-canary.20240328-7d2a420": "2024-03-28T03:16:26.510Z",
  "18.2.0-beta.3": "2024-03-28T16:36:22.952Z",
  "18.2.0": "2024-03-28T20:12:10.185Z",

@khludenevav
Copy link
Author

@xiongemi is there enough information to make a fix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xiongemi xiongemi

Labels
blocked: more info needed scope: core core nx functionality type: bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@FrozenPandaz @xiongemi @khludenevav