feat: add fp8 source files

[mickaelseznec]

No description provided.

[LucasWilkinson]

Thanks for the PR, I will try to test this soon!

[LucasWilkinson]

Apologies for the very long delay! the 12.1 build issues is now landed on main and the MLA stuff is landed to so we are cleared to land this.

However, is the hopper/flash_api.cpp changes strictly necessary? im really really trying to reduces diffs with upstream to make sure we can sync easily and this is any upstream file, is there any chance we could move this stuff in to vllm_flash_attn/flash_attn_interface.py and use torch.expand to create a 0 stride?

[mickaelseznec]

Great, I didn't know about torch.expand(). I'll use it on vllm side to avoid any magic hidden in the interface layer.

Also, I've seen some API mismatches between torch.ops._vllm_fa3_C.fwd and how it's called in vllm_flash_attn/flash_attn_interface.py (alibi doesn't seem to have support in FA3)

[LucasWilkinson]

Great, I didn't know about torch.expand(). I'll use it on vllm side to avoid any magic hidden in the interface layer.

Awesome, thank you! Do you mind removing these bits from the flash_api.cpp in this PR?

Also, I've seen some API mismatches between torch.ops._vllm_fa3_C.fwd and how it's called in vllm_flash_attn/flash_attn_interface.py (alibi doesn't seem to have support in FA3)

oh good catch! oof not sure how that slipped by, PRs welcome :smiling:

[mickaelseznec]

Hi @LucasWilkinson, I've updated my PR accordingly. Let me know if you have any further comments :)