Another way to approach the problem is to ask what is meant by "responsibility"; the following table might help elucidate some of the complexities of this matter:

(Exploring the notion of 'driver responsibility')

Moreover, although liability is frequently posited as an either/or proposition (“either the manufacturer or the driver”), it is rarely binary. In terms of just civil (that is, noncriminal) liability, in a single incident a vehicle owner could be liable for failing to properly maintain her vehicle, a driver could be liable for improperly using the vehicle’s automation features, a manufacturer could be liable for failing to adequately warn the user, a sensor supplier could be liable for poorly manufacturing a spotty laser scanner, and a mapping data provider could be liable for providing incorrect roadway data. Or not.

The relative liability of various actors will depend on the particular facts of the particular incident. Although automation will certainly impact civil liability in both theory and practice, merely asking “who is liable in tomorrow’s automated crashes” in the abstract is like asking “who is liable in today’s conventional crashes.” The answer is: "It depends."

Second, the only way to truly “solve” civil liability is to prevent the underlying injuries from occurring. If automated features actually improve traffic safety, they will help in this regard. As a technical matter, however, it is doubtful that requiring drivers merely to initiate lane changes will ensure that they are engaged enough in driving that they are able to quickly respond to the variety of bizarre situations that routinely occur on our roads. This is one of the difficult human-factors issues present in what I call the “mushy middle” of automation, principally SAE levels 2 and 3.

Science fiction writer Arthur C. Clarke expressed his opinion about the future of autonomous cars and the responsibility of human drivers in his 1976 novel :

As the beautiful old car cruised in almost perfect silence under the guidance of it's automatic controls, Duncan tried to see something of the terrain through which she was passing. The spaceport was 50 km from the city - no one had yet invented a noiseless rocket - and the four-lane highway bore a surprising amount of traffic. Duncan could count at least 20 vehicles of different types and even though they were all moving in the same direction, the spectacle was somewhat alarming.

"I hope all those other cars are on automatic," he said anxiously.

Washington looked a little shocked. "Of course," he said. "It's been a criminal offense for at least a hundred years to drive manually on a public highway. But we still have occasional psychopaths to kill themselves and other people..."
(Read more about Arthur C. Clarke's autonomous cars)

Update 07-Oct-2017: Isaac Asimov stated this idea a quarter-century earlier than Clarke; see the entry for laws against human drivers. End update.