Error: "LayerNormKernelImpl" not implemented for 'Half'

[dwrensha]

After 9299059, I get this error whenever I try to use the llmstep tactic:

process 'python3' exited with code 1

I tried adding this error logging:

diff --git a/python/server.py b/python/server.py
index 0efa17b..0322341 100644
--- a/python/server.py
+++ b/python/server.py
@@ -94,6 +94,7 @@ class LLMStepRequestHandler(BaseHTTPRequestHandler):
             response = result
             self.wfile.write(json.dumps(response).encode('utf-8'))
         except Exception as e:
+            print("Error", e)
             error_response = {'error': str(e)}
             self.wfile.write(json.dumps(error_response).encode('utf-8'))

And the error shown is:

Error "LayerNormKernelImpl" not implemented for 'Half'

The error goes away if I revert 9299059.

---

For additional context:
I'm running python python/server.py with no arguments. Additionally, I've disabled cuda because my card does not have enough memory:

diff --git a/python/server.py b/python/server.py
index 0efa17b..0c2c8b5 100644
--- a/python/server.py
+++ b/python/server.py
@@ -18,7 +18,7 @@ def load_hf(hf_model):
         model = transformers.AutoModelForCausalLM.from_pretrained(hf_model)
         tokenizer = transformers.AutoTokenizer.from_pretrained(hf_model)
 
-    if torch.cuda.is_available():
+    if False:
         model.cuda()
     model.eval()
     print("Done.")