Explainable AI for Autonomous Driving

An autonomous vehicle is a decision procedure whose interior is opaque even to its authors. The networks that drive it are accurate and unaccountable at once, and the two properties are not in tension by accident: capacity purchased through depth is capacity withdrawn from inspection. Acceptance of such agents turns on a narrower question than accuracy. When a car brakes, swerves, or declines to act, an operator, a passenger, and an accident investigator each require an answer to why, and the answer must be legible to them rather than to the optimizer. This team study surveys the explanation methods available to self-driving agents, subjects them to experiment on crash footage, and proposes where in the pipeline explanation ought to live.

The partition. Explainability in the driving stack is not one problem. It factors along the pipeline, and each factor admits a different species of answer:

• Perception turns camera, LiDAR, RADAR, and GPS returns into a representation of the scene. Its explanations are attributions over input: gradient methods (class saliency maps, Grad-CAM, DeConvNet, guided backpropagation), activation methods (CAM, attention branch networks, layer-wise relevance propagation), and perturbational methods (LIME, SHAP) that interrogate the model by occlusion.
• Localization places the vehicle inside that representation. The lineage runs from landmark-based maximum a posteriori estimation through the extended Kalman filter — convergent but brittle under compounding association error — to the graph-based formulation, where raw measurements become edges encoding transition distributions between candidate poses. Explanation here is attribution over sensors rather than pixels.
• Planning selects trajectories against a predicted cost over progress, comfort, safety, and fuel. It is the least explained stage of the stack; search-based planners compose motion, behavior, and mission planning in parallel, and almost no interpretive tooling exists for the composite.
• System management governs what the vehicle records about itself. Authenticity is served by a blockchain event recorder: vehicles within dedicated short-range communication form a federation, a lead verifier writes accident data to the chain, and an adversary must compromise a majority of community or federation in real time. Integrity is served by a smart black box built on deterministic memory machines with local buffer optimization, compressing gigabit-per-second onboard streams by data value rather than recency, with a priority queue evicting the cheap.

Attribution formalized. For a class score $y^c$ and the activation maps $A^k$ of a final convolutional layer, Grad-CAM weights each map by the spatially pooled gradient

and the rectified combination $L^c$ localizes the evidence for $c$ without altering the architecture — the property CAM lacks, being confined to networks without fully connected heads. For localization, the analogous instrument is the Shapley value over the sensor set $F$,

which prices the marginal contribution of LiDAR against GNSS against inertial measurement for a given pose estimate, averaged over every coalition of the remaining sensors.

Benchmarking on crash footage. The experimental substrate is the CarCrash Dataset, restricted to clips whose ego vehicle is directly involved so that every attribution is egocentric. Over a Faster R-CNN backbone, class activation mapping localizes cleanly when a single object owns the frame and degrades precisely where an investigator needs it: with several vehicles of one class present, the heat assigns credit to a neighboring car whose pixels dominate the class score while the detector's boxes track a smaller, nearer one — and when the frame truncates the salient vehicle, boxes fail to appear at all. A semantic segmentation backbone repairs the granularity, scoring each pixel within its own predicted class, though its maps decentralize as the subject closes on the egocentric view. Deep feature factorization proved the sturdiest of the visual methods, clustering the frame into abstract objects with no output labels supplied, and segmenting the consequential regions of the crash footage unsupervised.

Explaining a control output. PilotNet maps pixels to a steering command, which makes its explanations answer for an action rather than a label. Grad-CAM over the steering head attributed a recorded accident to the ego vehicle's own lane crossing; the same model, modified to accept live OpenCV input, met the failure mode that matters operationally — sun glare degrading attribution and perception alike, an argument for redundant explainers rather than any single one. ADAPT, a captioning transformer trained on the Berkeley DeepDrive eXplanation corpus, narrates and justifies each control decision in natural language; on crash clips its narration is accurate and myopic at once, describing the ego vehicle's motion faithfully while missing the holistic context of the collision.

Where the explanation stops transferring. The sharpest result is negative. PilotNet, trained on road video, was run against maritime data collected by the Dartmouth Robotics Lab — seventy gigabytes of ROSBAG recordings, two and a half hours of time-synchronized RGB and inertial steering signal. The transferred model failed outright; a model retrained on the marine data fared little better, holding near $3.0$ mean squared error against under $1.0$ in the car domain. Grad-CAM diagnosed the gap: road-domain saliency rests its mass on the lane boundary, and open water supplies no such feature. The attribution did not merely lose precision under domain shift; the causal anchor it had been resting on was absent from the new domain entirely.

Explaining a policy. The planning stage answers to a different formalism. An action-value satisfies the Bellman recursion over reward,

which states what the policy is worth while remaining silent on what the agent will do. Substituting a named feature $F$ for the reward yields the generalized value function,

the expected future course of a quantity a person chose: velocity, tilt angle, distance to goal, landing-leg state. Learned alongside the policy of an autonomous lunar lander, these functions trace each feature's trajectory as the agent descends, and the trace reads as a statement of intent where the plain $Q$ reads as a score.

The framework. Intrinsic and post-hoc explanations already fall out of each stage as by-products of prediction; the proposal is to stop discarding them. Each stage acquires a contingency model estimating how far its own prediction deserves trust, and that estimate travels forward with the prediction — so planning receives what perception saw together with a price on the seeing. Explanation becomes a first-class signal in the pipeline rather than an artifact rendered for the postmortem.

Scaling the instrumentation. Interpretability work intervenes on intermediate activations, and the prevailing tooling binds those interventions to a particular authorship of the network: hook systems presuppose the object-oriented module tree the model happened to be written as, while the methods themselves grow stranger and the models larger. As proof of concept, the study implements a system representing a model as a computational graph over which a user matches sub-graphs to arbitrary functions — collecting and manipulating activations with no reference to the model's internals or to the idioms of its construction.

The conclusions are the experiments' own. No single method explains the pipeline; the methods that read most clearly to a person are not the ones that scale; and an attribution is only as durable as the feature it rests on, which a change of domain may simply delete.