Our LLM-controlled office robot can't pass butter