Can anyone tell me what affect auto-scaling has on nonessential containers? Do nonessential containers run again when scaling?

The auto-scaler should essentially run

kubectl drain

to evict all pods from a node before scaling it down. Then the scheduler will relaunch those pods on different nodes.