Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x86: Fix issue with PACKUSWB when the value to convert is exactly 0x00ff #6514

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Sleigh-InSPECtor
Copy link
Contributor

The PACKUSWB instruction performs a lane-wise conversion of signed 16-bit integers to 8-bit integers, saturating the results if they are out-of-range (i.e., to the range 0 to 255).

In the SLEIGH implementation of the saturation operation, if the signed 16-bit integer is exactly equal to 0x00ff it will be converted to the byte 0x00 (instead of 0xff).

The issue with the value 0x00ff in particular is that that both (sword s> 0xff:2) and (sword s< 0xff:2) are false causing neither of the conditions. In the patch I choose to change the second condition to s<= (matching the comment above the macro), however changing the first condition would also fix the issue.

e.g.:

660f67db "PACKUSWB XMM3, XMM3" with XMM3=0x00ff_00ff

  • Hardware Reference (AMD CPU): { XMM3 = 0xff_ff_00_00_00_00_00_00_ff_ff }
  • x86:LE:64:default (Existing): { XMM3 = 0x0 }
  • x86:LE:64:default (This patch): { XMM3 = 0xff_ff_00_00_00_00_00_00_ff_ff }

(Note: The repetition of the bytes is correct because XMM3 is both the source and destination register).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature: Processor/x86 Status: Triage Information is being gathered
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants