Over 20% of all accidents happen due to distracted drivers, but it's not just phones. Just talking to your passenger makes you a distracted driver, as does fiddling with the car sound system, drinking something and any other activity that isn't driving. Lots and lots of people treat their cars like extensions of their homes and do stuff in there they shouldn't, like eat. Or shave. Or apply nail polish... there was a famous case of a nurse who killed a motorcyclist doing that, there was nail polish all over the car dashboard.
Bottom line is just that people have no business driving anything. Humans are just bad at it, even those few who take it seriously.