Yup, it is a real risk. But on a lighter side, it’s a risk that we [humanity] have been fighting against since forever - the possibility of some of us causing harm to the others not due to malice, but out of assumptiveness and similar character flaws. (In this case: “I assume that the AI is reliable enough for this task.”)