Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hg-bigbird-ci-failure] Change fail_accuracy to fail_to_run for export #126591

Closed
wants to merge 3 commits into from

Conversation

anijain2305
Copy link
Contributor

@anijain2305 anijain2305 commented May 17, 2024

Stack from ghstack (oldest at bottom):

I don't really know what made hf_Bigbird pass earlier, but it should never have. The model uses numpy in between the pytorch operations to introduce randomness. So it should have graph breaks. So, this is actually an IMPROVEMENT.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang

anijain2305 added a commit that referenced this pull request May 17, 2024
ghstack-source-id: c6845bffa4a4b668cd1f5e9b85b459c024ce9497
Pull Request resolved: #126591
Copy link

pytorch-bot bot commented May 17, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126591

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit 89b812d with merge base 93844a3 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…n for export"



I don't really know what made hf_Bigbird pass for export earlier, but it should never have. The model uses `numpy` in between the pytorch operations to introduce randomness.

[ghstack-poisoned]
anijain2305 added a commit that referenced this pull request May 18, 2024
ghstack-source-id: aef5438fdddac2727d4144712b51716886af1485
Pull Request resolved: #126591
@anijain2305 anijain2305 added the topic: not user facing topic category label May 18, 2024
@@ -138,7 +138,7 @@ hf_Bert_large,pass,0



hf_BigBird,fail_accuracy,0
hf_BigBird,fail_to_run,0
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It fails to run because this is export. So, fails_to_run is an improvement.

@@ -150,7 +150,7 @@ hf_Bert_large,pass,0



hf_BigBird,fail_to_run,0
hf_BigBird,pass,46
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely an improvement.

@@ -150,7 +150,7 @@ hf_Bert_large,pass,0



hf_BigBird,fail_to_run,0
hf_BigBird,fail_accuracy,46
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mehh ... not sure what category this is, but baseline is bad anyways.

@anijain2305 anijain2305 requested review from ezyang and huydhn May 18, 2024 03:55
…n for export"



I don't really know what made hf_Bigbird pass earlier, but it should never have. The model uses `numpy` in between the pytorch operations to introduce randomness. So it should have graph breaks. So, this is actually an IMPROVEMENT.

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx chenyang78 kadeng chauhang

[ghstack-poisoned]
anijain2305 added a commit that referenced this pull request May 18, 2024
ghstack-source-id: d53c9aa905965bb85442e64aa8080a2c63e68fe1
Pull Request resolved: #126591
@anijain2305
Copy link
Contributor Author

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label May 19, 2024
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: Command git -C /home/runner/work/pytorch/pytorch cherry-pick -x f57c0fdc0cf95603e8e70abca0c826e75b9f8c3d returned non-zero exit code 1

Auto-merging benchmarks/dynamo/ci_expected_accuracy/aot_eager_torchbench_training.csv
CONFLICT (content): Merge conflict in benchmarks/dynamo/ci_expected_accuracy/aot_eager_torchbench_training.csv
error: could not apply f57c0fdc0cf... [hg-bigbird-ci-failure] Update csv files
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Details for Dev Infra team Raised by workflow job

@anijain2305
Copy link
Contributor Author

Done at #126414

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request module: dynamo topic: not user facing topic category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants