[AOTI][UX] One has no way of knowing whether they need to load DSO as CPU or CUDA runner

[malfet]

🐛 Describe the bug

Right now Inductor API offers no way queuing whether given DSO target CUDA or CPU

And attempts to load CUDA binaries in CPU corrupts CUDA context, so trick like

try {
  runner = new torch::inductor::AOTIModelContainerRunnerCpu(model_path);
} catch (std::runtime_error& e) {
  runner = new torch::inductor::AOTIModelContainerRunnerCuda(model_path);
}

does not work (though if invoked in opposite order it does, but it might change in the future)

Two proposals:

• As right now AOTI is limited to one architecture it should simple check whether device it's being instantiated on matches the one it's been exported from
• In a longer term, one should have an API to query whether model was created for CUDA or CPU (or may be even for both)

Versions

2.3.0, nightly

cc @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang @desertfire @chenyang78

[malfet]

A very naive and simple approach of achieving something of the nature is #126536